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EXECUTIVE  SUMMARY 


This  report  summarizes  the  National  Convective  Weather  Forecast  (NCWF)  1999 
Assessment  conducted  at  Comair  and  Delta  Airlines  from  April  through  November  1999. 
Specific  results,  conclusions,  and  recommendations  for  the  assessment  are  detailed 
within  this  report. 

The  Research  Applications  Program  at  the  National  Center  for  Atmospheric  Research 
(NCAR/RAP)  has  developed  the  NCWF  to  provide  graphical  information  regarding  the 
current  and  short-term  forecast  location  of  thunderstorm  activity.  The  NCWF  combines 
radar  information  along  with  lightning  data  to  produce  a  convective  detection  field  as 
well  as  1-  and  2-hour  forecasts  of  convective  weather. 

The  Federal  Aviation  Administration  (FAA)  William  J.  Hughes  Technical  Center, 

Weather  Branch  (ACT-320)  conducted  an  assessment  of  the  NCWF.  The  assessment 
involved  aircraft  dispatch  personnel  at  Comair  and  Delta  Airlines,  working  in  their 
respective  operational  environments.  The  assessment  focused  on  the  value,  perceived 
benefit,  and  performance  of  the  NCWF  for  airline  dispatch  use,  in  addition  to  collecting 
information  on  enhancements  made  by  NCAR/RAP  from  previous  versions  of  the 
NCWF.  ACT-320  personnel  collected  user  feedback  through  a  variety  of  means 
including  questionnaires,  interviews,  operational  observations,  and  Internet-based  user 
logs. 

Assessment  results  indicated  the  NCWF  was  useful  to  airline  dispatch  operations.  The 
graphical  representation  of  both  real-time  and  forecast  convection  was  helpful.  The 
NCWF  was  useful  when  viewed  as  an  additional  piece  of  information  rather  than  as  a 
stand-alone  product.  However,  issues  regarding  the  forecast  accuracy,  especially  the 
2-hour  forecast,  lessened  the  NCWF’s  value. 

Dispatchers  identified  that  the  NCWF  provided  benefit  to  most  of  their  job  tasks  and 
decisions.  These  included  preparing  flight  plans,  planning  flight  routes,  avoiding  severe 
weather,  selecting  alternate  airports,  selecting  altitudes,  monitoring  flight  conditions, 
rerouting  flights,  and  advising  pilots  enroute  of  severe  convective  weather. 

Overall  readability  ratings  for  the  NCWF  graphical  interface  and  characteristics  were 
acceptable.  However,  larger  annotations,  less  clutter,  and  improved  zoom  functions 
were  identified  as  desired  improvements. 

From  the  dispatchers’  perspective,  the  accuracy  of  the  NCWF  forecasts  needs 
improvement.  While  it  was  identified  that  the  1  -hour  forecast  only  needed  minor 
improvements,  a  significant  portion  of  the  users  found  the  2-hour  forecast  unacceptable. 
It  was  noted  that  the  NCWF  appears  to  handle  large,  organized  storms  in  an  accurate 
manner;  however,  the  performance  with  smaller,  air  mass  storms  is  not  as  good. 

Several  enhancements  were  identified  that  would  further  increase  the  utility  of  the 
NCWF.  The  most  frequently  noted  enhancements  included  animation,  growth  and 


decay  of  convective  areas,  larger  graphics  and  text,  improved  zoom  capability,  and 
additional  overlays. 

The  assessment  demonstrated  the  utility  of  the  NCWF  for  airline  dispatch  operations. 
However,  further  development  should  concentrate  on  improved  accuracy  of  the 
forecasts,  especially  the  2-hour  forecast.  Convective  growth  and  decay  capabilities 
should  also  be  incorporated. 
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1.  INTRODUCTION. 


1.1  BACKGROUND. 

Thunderstorms  are  a  cause  of  national  airspace  delays  during  the  summer  season. 
Thunderstorm  phenomena  such  as  lightning,  turbulence,  hail,  icing,  and  poor  visibility 
require  airspace  users  to  provide  a  wide  margin  of  separation  between  these 
phenomena  and  aircraft  operations.  In  order  to  help  avoid  these  phenomena,  reliable 
detection  and  forecasts  of  convective  activity  are  required. 

In  response  to  the  need  for  detection  and  forecasts,  the  Federal  Aviation  Administration 
(FAA)  Aviation  Weather  Research  Program  (AWRP)  has  sponsored  the  development  of 
prototype  convective  weather  products  for.both  the  terminal  and  enroute  environments. 
For  the  enroute  environment,  the  National  Center  for  Atmospheric  Research  (NCAR) 
Research  Applications  Program  (RAP),  working  as  part  of  the  AWRP  Convective 
Weather  Product  Development  Team,  has  developed  the  National  Convective  Weather 
Forecast  (NCWF).  The  NCWF  provides  graphical  information  regarding  the  current  and 
short-term  forecast  location  of  thunderstorm  activity. 

The  FAA  William  J.  Hughes  Technical  Center  Weather  Branch  (ACT-320)  previously 
assessed  two  developmental  versions  of  the  NCWF.  The  first  was  assessed  during  the 
summer  of  1997,  as  the  Automated  Convective  SIGMET  Forecast  (ACSF)  Product. 

The  ACSF  provided  a  1 -hour  forecast  of  convective  activity.  During  the  assessment, 
convective  forecasters  at  the  National  Oceanic  and  Atmospheric  Administration  (NOAA) 
Aviation  Weather  Center  (AWC)  used  the  product  as  a  tool  to  aid  in  creating  Convective 
Significant  Meteorological  Statements  (SIGMET).  Results  revealed  that  some 
components  of  the  product  had  merit;  however,  specific  improvements  would  be 
required  in  order  to  make  it  useful  for  SIGMET  generation.  Assessment  results  are 
documented  in  the  Automated  Convective  SIGMET  Forecast  Product  Demonstration 
Summary  Report,  December  1997  (available  from  ACT-320). 

The  second  assessment  of  the  product,  renamed  the  NCWF,  occurred  during  the 
summer  of  1 998,  and  involved  a  usability  study  using  aircraft  dispatch  personnel  from 
two  major  and  one  regional  airlines:  (1)  Delta  Airlines,  (2)  Northwest  Airlines,  and 
(3)  Atlantic  Coast  Airlines,  respectively.  Overall  results  indicated  that  the  NCWF  was 
valuable  in  identifying  and  forecasting  convective  weather;  was  useful  in  performing  job 
tasks;  and  performed  well,  particularly  with  well-defined  frontal  weather  situations.  In 
addition,  potential  improvements  and  enhancements  were  identified.  ACT-320 
documented  the  results  in  the  National  Convective  Weather  Forecast  (NCWF)  Product 
Demonstration  Final  Report,  October  1999. 

The  1999  iteration  of  the  NCWF  included  interface  and  algorithm  changes,  most 
notably,  interactive  zoom  capabilities  and  the  addition  of  a  2-hour  forecast.  In  order  to 
ascertain  the  value  and  performance  of  the  updated  NCWF,  a  third  assessment  was 
conducted.  This  third  operational  assessment  included  airline  dispatchers  from  Comair 
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and  Delta  Airlines  utilizing  the  NCWF  at  their  respective  airline  dispatch  operation 
centers.  The  1999  assessment  is  the  focus  of  this  report. 

1.2  PURPOSE  OF  REPORT. 

The  purpose  of  this  report  is  to  document  the  results  of  the  1999  NCWF  Assessment 
concerning  the  value  and  performance  of  the  NCWF  for  airline  dispatch  use. 

1.3  SCOPE. 

This  report  will  summarize  assessment  conduct  and  results.  The  report  will  describe 
the  NCWF,  assessment  objectives,  methodology,  location,  participants,  results, 
conclusions,  and  recommendations.  The  scientific  and  meteorological  aspects 
(including  accuracy)  of  the  NCWF  are  being  addressed  separately  by  NCAR  and  are 
not  a  part  of  this  report. 

2.  REFERENCE  DOCUMENTS. 

a.  FAA  Acquisition  Management  System  Test  and  Evaluation  Process  Guidelines, 
June  1999. 

b.  FAA-Standard-024B,  Content  and  Format  Requirements  for  the  Preparation  of 
Test  and  Evaluation  Documentation,  August  22, 1994. 

c.  Automated  Convective  SIGMET  Forecast  (ACSF)  Product  Demonstration 
Summary  Report,  December  19,  1997;  ACT-320. 

d.  National  Convective  Weather  Forecast  (NCWF)  Product  Demonstration  Final 
Report,  October  19,  1999;  ACT-320. 

e.  National  Convective  Weather  Forecast  (NCWF)  Assessment  Plan,  June  4,  1999; 
ACT-320. 


f.  National  Convective  Weather  Forecast  (NCWF)  Assessment  Procedures, 

July  15,  1999;  ACT-320. 

3.  SYSTEM  DESCRIPTION. 

3.1  SYSTEM  OVERVIEW. 

3.1.1  NCWF  Operations. 

The  NCWF  software  ran  at  NCAR/RAP.  Output  was  produced  every  5  minutes  and 
placed  on  the  NCAR  web  server.  Access  to  the  NCWF  was  via  the  Internet  using  the 
web  address  of  http://ncwf.rap.ucar.edu.  NCAR/RAP  maintained  the  running  of  the 
NCWF;  however,  since  they  are  not  a  24-hour  per  day,  7-day  per  week  facility,  it  was 
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recognized  that  reliability  could  be  an  issue,  especially  on  weekends  and  during  night 
hours. 

Comair  dispatchers  viewed  the  NCWF  on  dedicated  computers  located  in  the 
dispatcher  work  areas  while  participating  Delta  Airlines  dispatchers  used  personal 
laptop  computers.  In  order  for  the  NCWF  to  be  viewed  correctly  either  Netscape  4.06  or 
Internet  Explorer  4.0  or  higher  was  necessary  as  the  Internet  browser.  The  1998 
version  of  the  NCWF  could  be  viewed  with  Netscape  3.0  or  Internet  Explorer  3.0  or 
higher. 

3.1.2  Detection  Field. 

The  NCWF  determines  the  locations  of  convective  weather  from  the  integration  of 
lightning  and  radar  data.  The  data  sources  for  the  NCWF  are  the  Composite  Base 
Reflectivity  and  Echo  Top  Mosaics  from  Weather  Services  International  (WSI),  and 
National  Lightning  Detection  Network  (NLDN)  data  from  Global  Atmospherics,  Inc. 
received  via  Kavouras.  This  data  goes  through  numerous  steps  before  becoming  the 
final  NCWF  detection  field:  a  two-dimensional  4  kilometers  (km)  spatial  resolution 
convective  field  that  updates  automatically  every  5  minutes. 

The  chronological  steps  in  determining  the  detection  field  are  summarized  in  the 
following  paragraphs.  NCAR/RAP  should  be  contacted  for  more  detailed  information. 

a.  The  2-km  National  Composite  Base  Reflectivity  and  4-km  Echo  Tops  from  WSI 
are  converted  into  a  4-km  resolution  grid  field. 

b.  A  filter  process  removes  data  with  echo  tops  less  than  1 7,000  feet  in  order  to 
eliminate  most  ground  clutter  and  nonconvective  activity. 

c.  A  convection  detection  field  value  between  1  and  6  is  assigned  to  each  grid  point 
based  upon  the  remaining  reflectivity  data  and  lightning  strike  information.  Lightning 
information  is  used  to  either  increase  the  detection  field  value  of  existing  reflectivity 
areas  or  to  add  areas  that  are  without  any  reflectivity  data. 

3.1.3  Forecast  Products. 

After  obtaining  the  detection  field,  another  series  of  steps  are  taken  to  identify 
potentially  long-lived  storms  and  to  provide  these  areas  with  1-  and  2-hour  forecasts 
depending  upon  the  following  criteria. 

a.  A  scale  separation  filter  is  applied  to  the  detection  field  to  eliminate  small-scale 
features  that  are  perishable  within  an  hour.  This  allows  the  tracking  of  the  storm 
envelope  Instead  of  individual  cells. 

b.  A  tracking  algorithm.  Thunderstorm  Identification  Tracking  and  Nowcasting 
(TITAN),  runs  on  the  filtered  field  to  determine  storm  motion.  TITAN  defines  a  storm 
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based  on  contiguous  regions  and  uses  a  centroid  tracker  technique  for  providing 
forecasts  based  upon  storm  track  and  trend. 

c.  The  storm  system  must  meet  all  of  the  following  thresholds  before  a  1-hour 
forecast  is  produced: 

1 .  Detection  field  level  3  or  greater; 

2.  Area  coverage  of  520  km^  or  greater; 

3.  Existed  for  at  least  45  minutes. 

d.  An  additional  filter  is  applied  to  the  individual  1  -hour  forecasts  to  determine  if  a 
2-hour  forecast  should  be  calculated.  This  filter  is  based  upon  the  maximum  average 
and  standard  deviation  of  the  detection  fields.  Larger  and  more  intense  storms  are 
characterized  by  higher  maximum  averages  but  lower  standard  deviations.  These 
storms  receive  a  2-hour  forecast  based  upon  TITAN.  The  larger  storms  are  more  likely 
to  be  accurately  forecasted  by  the  NCWF. 

It  should  be  noted  that  a  NCWF  convective  area  without  a  forecast  does  not  mean  the 
area  is  not  moving  or  will  not  persist  for  1  or  2  hours.  Some  convective  regions  may  not 
meet  one  or  more  of  the  above  stated  criteria  for  producing  a  forecast. 

3.2  NCWF  COMPONENTS  AND  FEATURES. 

Figure  1  illustrates  an  example  of  the  NCWF.  Various  components  and  features  of  the 
system  are  described  in  the  following  sections. 

3.2.1  Components. 

3.2.1. 1  Initial  Detection  Field. 

The  Initial  Detection  Field  (figure  1,  item  1)  displays  the  convective  intensity  based  on 
the  comparison  of  the  base  reflectivity  and  lightning  rate  fields.  This  field  is  displayed 
using  a  6-level  intensity  color  scale.  The  field  may  be  toggled  on  and  off  by  clicking  the 
Detection  field  box  in  the  Products  section  at  the  top  of  the  display. 

3.2.1 .2  Convective  Intensity  Color  Scale. 

The  Convective  Intensity  Color  Scale  (figure  1,  item  2),  located  on  the  right  of  the 
display,  indicates  the  intensity  level  of  the  convective  detection  field  with  the  highest 
levels  (5-6)  as  red;  moderate  levels  (3  and  4)  as  yellow  and  orange,  respectively;  and 
lowest  intensity  levels  (1  -2)  as  green.. 


4 


5 


FIGURE  1.  NCWF  DISPLAY 


3.2.1 .3  Forecast  Contours. 


Forecast  Contours  (figure  1 ,  item  3)  are  represented  as  blue  (1  hour)  and  black  (2  hour) 
colored  polygons.  Forecast  contours  may  be  toggled  on  or  off  by  clicking  the 
appropriate  box  in  the  Products  section  at  the  top  of  the  display. 

3.2.1. 4  Echo  Speed. 

Echo  Speed  (figure  1 ,  item  4)  is  annotated  in  knots  over  the  detection  areas  that  have 
forecast  contours.  The  Echo  Speed  precedes  the  Storm  Top  information  (section 
3.2.1 .5).  Echo  Speed  may  be  toggled  on  and  off  by  clicking  the  box  labeled  “Storm 
speed/top  annotations”  in  the  Products  section  at  the  top  of  the  display. 

3.2.1 .5  Storm  Tod. 

Storm  Top  (figure  1,  item  5)  is  annotated  in  hundreds  of  feet  following  the  Echo  Speed. 
Storm  Top  is  toggled  on  and  off  along  with  the  Echo  Speed  (see  section  3.2.1 .4). 

3.2.1. 6  Forecast  Direction  Lines. 

Forecast  Direction  Lines  (figure  1 ,  item  6)  are  displayed  as  white  lines  or  vectors, 
indicating  the  direction  of  storm  movement.  These  lines  are  displayed  only  when  the 
forecast  contours  are  displayed. 

3.2.2  Features. 

3.2.2.1  Views. 

Users  may  select  a  national  view  to  observe  the  contiguous  United  States,  or  they  may 
zoom  in  on  smaller  regions  by  using  the  mouse  to  draw  a  selection  box  around  the  area 
of  interest.  The  user  may  return  to  the  previous  zoom  or  national  zoom  by  clicking  the 
appropriate  buttons  located  in  the  upper  left  corner  of  the  display. 

3.2.2.2  Overlays. 

Overlays  on  the  NCWF  include  airports  (using  the  3-letter  identifier),  range  rings  (figure 
1 ,  item  7)  and  Air  Route  Traffic  Control  Center  (ARTCC)  boundaries  (figure  1 ,  item  8). 
The  airport  identifiers  also  include  selected  Navigation  Aid  (NAVAID)  sites,  for  example. 
Very  High  Frequency  (VHF)  OmniRange  Navigation  System  (VOR)  sites.  The  range 
rings  encompass  a  radius  of  50  nautical  miles  (nmi)  around  major  airports.  Due  to  the 
map  projection  used  for  the  NCWF,  the  range  rings  appear  as  slight  ovals,  rather  than 
as  exact  circular  rings.  The  ARTCC  boundaries  delineate  areas  of  responsibility  for 
each  of  the  individual  centers.  The  airports  and  range  rings  are  displayed  with  each 
other  and  may  be  toggled  on  and  off,  while  the  ARTCC  boundaries  may  be  separately 
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toggled  on  and  off  using  the  boxes  provided  at  the  top  of  the  display.  State  boundaries 
are  a  permanent  overlay. 

3.2.3  NCWF  Changes  for  1999. 

Changes  from  earlier  versions  of  the  NCWF  to  the  1999  NCWF  were  based  on  prior 
user  feedback  as  well  as  scientific  improvements  to  the  algorithm.  Changes  and 
enhancements  included  the  following: 

a.  Use  of  composite  reflectivity  data  rather  than  Vertically  Integrated  Liquid  (VIL) 
data  previously  used. 

b.  Addition  of  a  2-hour  forecast,  with  the  option  of  users  selecting  either  the  1-  or 
2-hour  forecast  option. 

c.  Smoothing  of  forecast  polygons. 

d.  User  selected  zoom  capability  replacing  preset  selections  for  national,  regional, 
and  ARTCC  views. 

e.  More  rapid  graphical  updates. 

f.  User  selected  overlay  options  of  ARTCC  boundaries,  airports,  and  50  nmi  range 
rings. 

4.  ASSESSMENT  DESCRIPTION. 

4.1  ASSESSMENT  OBJECTIVES. 

The  NCWF  assessment  was  conducted  to  assess  the  value,  perceived  benefit,  and 
performance  of  the  latest  version  of  the  NCWF  for  airline  dispatch  use.  Specific 
objectives  were: 

a.  Assess  the  value  of  the  NCWF  in  identifying  and  forecasting  convective  weather 
for  airline  dispatch  use. 

b.  Identify  NCWF  benefit  areas  for  dispatcher  tasks  and  decisions. 

c.  Assess  product  graphical  interfaces  and  characteristics. 

d.  Assess  NCWF  performance  and  accuracy  from  the  dispatchers’  perspective. 

e.  Identify  enhancements. 
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4.2  ASSESSMENT  ACTIVITIES. 


4.2.1  Assessment  Participants. 

4.2.1. 1  Delta  Airlines. 

Delta  Airlines  is  one  of  the  country’s  largest  airlines,  employing  over  140  dispatchers  in 
their  operations  center.  Their  route  structure  covers  the  contiguous  United  States. 
NCWF  Assessment  participants  were  a  sub-group  of  dispatchers  who  had  access  to  the 
1999  version  of  the  NCWF  (see  section  4.2. 2. 2).  Delta  Airlines  dispatch  operations  are 
supported  by  a  staff  of  company  meteorologists  with  access  to  considerable  weather 
information.  Delta  was  a  participant  in  the  1998  assessment. 

4.2. 1.2  Comair. 

Comair  is  one  of  the  largest  regional  airlines  and  operates  nearly  700  daily  flights 
covering  the  eastern  half  of  the  country.  A  staff  of  approximately  50  dispatchers  are 
employed  in  their  operations  center.  All  Comair  dispatchers  were  able  to  participate  in 
the  NCWF  assessment.  Unlike  Delta  Airlines,  Comair  does  not  maintain  its  own 
meteorology  staff.  Regional  airlines  typically  lack  the  myriad  weather  information 
sources  available  to  dispatchers  in  most  major  airlines.  Comair  was  not  a  participant  in 
previous  NCWF  assessments. 

4.2.2  Pre-assessment  Conduct  Activities. 

4.2.2. 1  Baseline  Measures. 

ACT-320  personnel  through  the  use  of  interviews  and  observations  identified  current 
procedures  and  information  sources  used  by  dispatchers  at  Comair  in  detecting  and 
forecasting  convective  weather.  The  information  was  documented  (see  appendix  A) 
and  provided  a  basis  for  comparison  between  current  methodologies  and  use  of  the 
NCWF.  The  baseline  data  was  collected  June  1-2, 1999.  In  order  to  avoid  bias,  the 
baseline  data  was  collected  before  introduction  of  the  NCWF  to  Comair  users. 

Baseline  information  for  Delta  Airlines  was  collected  for  the  1998  NCWF  Assessment. 
There  was  no  need  to  collect  additional  information  for  the  1999  assessment. 

4.2.2.2  Product  Implementation. 

In  order  for  Comair  dispatchers  to  view  the  NCWF,  NCAR  personnel  provided  and 
installed  four  dedicated  computers  in  the  dispatcher  operations  area.  Each  of  the 
computers  was  attached  to  an  Internet  Service  Provider  for  viewing  of  the  NCWF  on  an 
NCAR  developed  web  page.  The  computer  monitors  were  in  viewing  distance  from 
each  dispatcher's  individual  work  position.  Since  displays  were  shared,  the  displays 
could  be  turned  toward  particular  dispatchers  and  away  from  others.  In  addition. 
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dispatchers  had  to  share  a  keyboard  and  mouse  for  each  display,  thereby  limiting 
individual  interactive  capability. 

Delta  Airlines  had  Internet  access  via  its  Local  Area  Network.  However,  the  Internet 
browser  used  at  Delta  was  unable  to  display  the  1999  version  of  the  NCWF.  Thus,  it 
was  decided  that  a  sub-group  of  10  dispatchers  (Subject  Matter  Experts)  would  be 
utilized  for  formal  feedback.  This  group  of  dispatchers  used  personal  laptop  computers 
at  their  dispatcher  work  locations  with  an  appropriate  Internet  browser  to  allow  them  to 
view  the  1999  NCWF.  In  addition,  two  dispatcher  positions  (for  Delta  Express  flights) 
had  company-provided  computers  with  the  ability  to  display  the  1999  NCWF.  All  other 
Delta  dispatchers  only  had  access  to  the  1998  version  of  the  NCWF  and  were  not 
included  in  the  collection  of  feedback. 

4.2.2.3  Product  Training. 

All  product  training  was  provided  by  NCAR.  On-site  training  was  provided  to  the  Delta 
Airline  dispatchers  participating  in  the  assessment.  On-site  individual  training  of 
dispatchers  plus  a  train-the-trainer  approach  of  key  Comair  personnel  accomplished 
Comair  training. 

4.2.2.4  Product  Shakedown. 

After  product  activation  at  both  Delta  Airlines  and  Comair,  time  was  allocated  for 
product  shakedown  to  ensure  proper  meteorological  and  web  page  performance.  This 
also  provided  users  with  additional  product  familiarization  prior  to  official  assessment 
conduct.  Delta  Airlines  had  approximately  2  months  for  shakedown  while  Comair  had 
approximately  1  month  due  to  the  additional  effort  of  computer  installation. 

4.2.3  Assessment  Conduct. 

The  actual  assessment  of  the  NCWF  began  in  early  August  and  proceeded  until  the  first 
week  of  October  (approximately  2  full  months).  Assessments  at  both  sites  (Delta 
Airlines  and  Comair)  ran  concurrently.  Dispatchers  used  the  NCWF  operationally 
during  normal  work  shifts.  ACT-320  personnel  collected  user  feedback  through  a 
variety  of  tools  (see  section  4.3)  including  questionnaires,  interviews,  operational 
observations,  and  user  comment  logs. 

4.2.4  Assessment  Schedule  and  Locations. 

The  NCWF  Assessment  was  conducted  from  April  through  November  1999.  This 
period  included  planning,  preparation,  coordination,  and  conduct.  Planning, 
preparation,  and  coordination  were  primarily  conducted  at  ACT-320  and  NCAR  facilities 
in  Atlantic  City,  New  Jersey;  and  Boulder,  Colorado,  respectively.  Assessment  conduct 
occurred  at  Delta  Airlines’  operations  center  in  Atlanta,  Georgia,  and  Comair’s 
operations  center  at  the  Cincinnati  International  Airport,  Covington,  Kentucky.  Table  1 
provides  an  overall  assessment  schedule. 
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TABLE  1 .  NCWF  ASSESSMENT  SCHEDULE 


Activity 

Point  of  Contact 

Date 

Pre-assessment  briefing  at  Comair 

ACT-320,  NCAR 

April  28,  1999 

Baseline  data  collection  at  Comair 

ACT-320 

June  1-2,  1999 

Briefing  and  training  at  Delta 

ACT-320,  NCAR 

June  7-9,  1999 

Shakedown  at  Delta 

NCAR 

June  9  -  August  6, 1999 

Product  installation  at  Comair 

NCAR 

July  10-11,  1999 

Training  for  Comair 

NCAR 

July  12-14,  1999 

Shakedown  at  Comair 

NCAR 

July  15  -  August  6, 1999 

Assessment  conduct 

ACT-320 

August  7  -  October  31 , 1 999 

On-line  log  available  to  Delta 

ACT-320 

August  7  -  October  7, 1999 

Comair  observations  and  start  user  log 

ACT-320 

August  10-11, 1999 

On-site  interviews  at  Delta 

ACT-320 

August  23-24, 1999 

Telephone  interviews  with  Delta 

ACT-320 

September  9, 1 999 

Questionnaire  administered  to  Comair 

ACT-320 

October  6-7, 1999 

On-line  questionnaire  to  Delta 

ACT-320 

October  8-31, 1999 

Debrief  to  NCAR  and  Comair 

ACT-320 

November  1 , 1 999 

4.3  DATA  COLLECTION  AND  ANALYSIS  METHODS. 

NCWF  objectives  were  measured  by  obtaining  feedback  from  airline  dispatchers  at 
Delta  and  Comair  airlines.  Data  collection  methods,  described  in  the  following  sections, 
are  documented  in  the  NCWF  Assessment  Procedures,  July  15,  1999. 

4.3.1  User  Logs. 

User  logs  were  employed  to  record  dispatcher  comments  on  a  daily  basis  during  the 
assessment  period.  The  logs  were  designed  to  help  identify  specific  instances  of 
product  performance  along  with  the  date  and  time  of  the  occurrence.  Log  entries  were 
also  used  to  formulate  interview  and  questionnaire  material. 

Delta  Airline  dispatchers  used  an  on-line  log  that  was  developed  by  ACT-320  personnel 
(see  appendix  B).  Dispatchers  were  able  to  complete  the  log  by  providing  information 
on  the  perceived  accuracy  of  the  NCWF,  ease  of  use,  effect  on  dispatch  tasks,  and 
additional  comments.  At  the  end  of  log  completion,  dispatchers  were  able  to  submit 
their  completed  comments  to  ACT-320  electronically.  While  the  on-line  format  was 
appropriate  for  Delta,  paper  logs  were  developed  for  use  by  Comair  dispatchers  (see 
appendix  C).  While  Comair  dispatchers  had  access  to  the  Internet  for  viewing  the 
NCWF,  the  availability  of  only  4  computers  for  the  approximately  10  dispatchers  on  shift 
at  any  one  time,  made  access  to  the  Internet  for  log  completion  impractical.  Thus, 
paper  logs  were  prepared  and  distributed  to  each  dispatcher  work  location.  Comair 
dispatchers  completed  the  logs  by  providing  handwritten  comments  on  the  performance 
and  use  of  the  NCWF.  A  Comair  representative  collected  completed  logs  and  provided 
them  via  Federal  Express  to  ACT-320. 
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4.3.2  Operational  Observations. 


ACT-320  personnel  scheduled  operational  observations  at  Comair  on  August  10-11, 
1999.  The  intent  was  to  observe  dispatchers’  use  of  the  NCWF  during  periods  of 
convective  weather.  The  observations  were  designed  to  be  nonintrusive  to  dispatch 
operations  and  were  coordinated  with  dispatch  management  prior  to  the  actual  visit. 
Observation  logs  were  developed  to  record  how  and  under  what  circumstances  the 
product  was  used,  whether  other  products  were  used,  and  any  feedback  from  the 
observed  dispatcher.  Although  convective  weather  was  anticipated,  none  occurred. 
Therefore,  observational  data  was  not  obtained.  However,  discussions  with  a  number 
of  dispatchers  about  the  NCWF  were  conducted.  Results  of  these  discussions  were 
summarized  and  included  with  information  obtained  by  other  methods. 

4.3.3  Dispatcher  Interviews. 

ACT-320  personnel  conducted  on-site  and  telephone  interviews  with  Delta  Airlines 
dispatchers.  The  intent  of  the  on-site  interviews  was  to  solicit  information  that  was  not 
being  received  via  the  user  logs.  A  total  of  seven  dispatchers  were  interviewed  on-site 
to  help  determine  dispatcher  use  of  the  NCWF.  In  addition,  phone  interviews  were 
conducted  when  ACT-320  personnel  observed  convective  weather  that  may  have  been 
impacting  Delta  flights.  ACT-320  personnel  monitored  the  weather  by  utilizing  an 
Aircraft  Situation  Display  (ASD)  configured  to  display  Delta  Airline  flights  overlaid  with  a 
national  weather  radar  mosaic.  Telephone  interview  questions  are  located  in 
appendix  D. 

4.3.4  Questionnaires. 

ACT-320  personnel  developed  a  questionnaire  that  was  administered  to  both  Delta 
Airlines  and  Comair  dispatchers  at  the  end  of  the  assessment  phase.  The  purpose  of 
the  questionnaire  was  to  rate  the  utility,  readability,  and  accuracy  of  the  NCWF 
components  and  features,  and  rate  the  usefulness  of  the  NCWF  for  dispatcher  job 
tasks,  in  addition,  open-ended  questions  provided  dispatchers  the  opportunity  to 
provide  additional  comments,  suggestions  for  improvements,  and  enhancements.  A 
copy  of  the  questionnaire  is  included  in  appendix  E.  The  questionnaire  was 
administered  electronically  via  the  ACT-320  web  page  to  Delta  participants,  while 
Comair  participants  were  administered  paper  copies.  Most  of  the  Comair  responses 
were  provided  during  a  visit  by  ACT-320  personnel  who  oversaw  the  administration  of 
the  questionnaire  and  were  available  for  clarifications  or  questions. 
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4.3.4. 1  Utility,  Readability,  and  Accuracy. 

As  part  of  the  questionnaire,  dispatchers  were  asked  to  rate  the  NCWF  on  the  three 
dimensions  of  utility,  readability,  and  accuracy  using  a  5-point  Likert  scale.  NCWF 
components  and  features  were  rated  on  all  or  a  subset  of  the  three  dimensions.  Items 
rated  were: 

a.  Initial  Detection  Fields 

b.  Convective  Intensity  Color  Scale 

c.  Detection  Field  Colors 

d.  1  -Hour  Forecast  Contours 

e.  2-Hour  Forecast  Contours 

f.  Forecast  Direction  Lines 

g.  50  nmi  Range  Rings 

h.  Airport  Identifiers 

i.  Storm  Top  Information 

j.  Echo  Speed 

k.  ARTCC  Boundary  Overlays 

l.  User  Selected  Zoom 

m.  NCWF  Update  Rate 

The  rating  scale  provided  to  the  dispatchers  consisted  of  the  following: 

a.  1  =  Largely  Acceptable.  This  response  indicates  the  NCWF  component  being 
assessed  consistently  enhances  your  ability  to  meet  the  requirements  of  your  job;  likely 
to  lead  to  enhanced  job  performance. 

b.  2  =  Acceptable.  This  response  indicates  the  NCWF  component  being  assessed 
frequently  enhances  your  ability  to  meet  the  requirements  of  your  job;  may  lead  to 
enhanced  job  performance. 

c.  3  =  Borderline.  This  response  indicates  that,  although  the  NCWF  component 
being  assessed  is  adequate,  minor  improvements  would  make  it  more  helpful  in 
identifying  and  forecasting  convective  weather,  and  does  not  degrade  or  improve  job 
performance. 

d.  4  =  Unacceptable.  This  response  indicates  the  NCWF  component  being 
assessed  frequently  impedes  your  ability  to  meet  the  requirements  of  your  job;  may  lead 
to  degradation  of  job  performance. 

e.  5  =  Largely  Unacceptable.  This  response  indicates  the  NCWF  component  being 
assessed  consistently  impedes  your  ability  to  meet  the  requirements  of  your  job;  likely 
to  lead  to  degradation  of  job  performance. 

Following  the  rating  questions,  open-ended  questions  were  used  to  solicit  comments  as 
well  as  suggestions  for  improvement. 
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4.3A.2  Usefulness  to  Tasks. 


Another  aspect  of  the  questionnaire  involved  dispatcher  assessment  of  NCWF 
usefulness  in  relation  to  operational  tasks.  Tasks  were  identified  via  baseline  data 
collection  and  verified  by  dispatch  management.  A  5-point  Likert  scale  was  used  to 
measure  NCWF  usefulness  regarding  the  following  specific  tasks: 

a.  Preparing  flight  plans, 

b.  Planning  fuel  quantity, 

c.  Planning  flight  route, 

d.  Avoidance  of  severe  convective  weather, 

e.  Selecting  an  alternate  airport, 

f.  Selecting  altitude, 

g.  Monitoring  flight  conditions, 

h.  Re-routing  flights, 

i.  Advising  pilots  enroute  of  severe  convective  weather, 

j.  Anticipating  ground  delays,  and 

k.  Anticipating  Air  Traffic  Control  (ATC)  actions  (e.g.,  rerouting,  closing  runways, 
changing  arrival  gates). 

The  rating  scale  included  the  following: 

a.  1  =  Of  Considerable  Use.  This  response  indicates  the  NCWF  had  a  significant 
positive  effect  on  decisions  and  activities  related  to  the  performance  of  this  task. 

b.  2  =  Somewhat  Useful.  This  response  indicates  the  NCWF  had  a  positive  effect 
on  decisions  and  activities  related  to  the  performance  of  the  task. 

c.  3  =  Borderline.  This  response  indicates  the  NCWF  had  little  to  no  effect  on 
decisions  and  activities  related  to  the  performance  of  the  task. 

d.  4  =  Of  Little  Use.  This  response  indicates  the  NCWF  had  a  negative  effect  on 
decisions  and  activities  related  to  the  performance  of  the  task. 

e.  5  =  Not  at  all  Useful.  This  response  indicates  the  NCWF  had  a  significant 
negative  effect  on  decisions  and  activities  related  to  the  performance  of  the  task. 

4.3.5  Data  Analysis. 

Much  of  the  feedback  from  the  user  logs,  observations,  interviews,  and  questionnaire 
was  in  the  form  of  comments,  answers  to  open-ended  questions,  and  interview 
responses.  These  responses  were  recorded  and  are  summarized  in  the  results  section 
(section  5). 
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Since  response  to  the  user  logs  was  limited  (see  section  5.1),  the  majority  of  the 
assessment  results  are  based  on  questionnaire  responses.  Questionnaire  data  was 
analyzed  using  descriptive  statistics.  Rating  scores  data  is  ordinal  and  is  not  normally 
distributed;  therefore,  the  median  was  used  as  a  measure  of  central  tendency.  The 
median  is  the  value  above  and  below  which  one  half  of  the  observations  fall.  When 
there  is  an  even  number  of  observations,  no  unique  center  value  exists,  so  the  mean  of 
the  two  middle  observations  is  taken  as  the  median  value.  For  the  questionnaire  rating 
data,  low  median  values  (near  1  and  2)  indicated  generally  positive  feedback,  while  high 
values  (4  and  5)  indicated  negative  feedback.  Rating  scale  response  percentages  were 
also  calculated. 

5.  RESULTS. 

5.1  FACTORS  AFFECTING  RESULTS. 


a.  Small  Response  Set:  The  number  of  questionnaire  responses  from  Delta  airlines 
was  limited.  Even  though  Delta  participation  was  limited  to  10  Subject  Matter  Experts, 
only  5  of  these  participants  responded.  Whether  these  responses  are  representative  of 
Delta,  or  airline  dispatchers  in  general,  is  questionable,  as  this  number  may  not  be 
considered  statistically  significant.  Additionally,  Delta  did  not  provide  any  user  log 
responses  noting  specific  instances  of  product  performance.  Based  on  this  limited 
feedback,  it  is  difficult  to  ascertain  Delta’s  actual  opinion  of  the  NCWF. 

b.  User  log  responses  were  limited  for  both  Delta  and  Comair:  As  noted  above. 
Delta  did  not  provide  any  user  log  responses,  while  Comair  provide  only  1 1  log 
responses  out  of  a  total  dispatcher  population  of  approximately  40  individuals  working 
7  days  a  week  over  the  nearly  2-month  period  when  the  user  log  was  available.  Thus, 
most  of  the  results  of  the  assessment  are  based  on  rating  information  from  the 
questionnaire,  supplemented  by  comments. 

c.  Lack  of  Operationally  Significant  Convective  Weather:  According  to  dispatchers 
and  product  developers,  significant  convective  weather  activity  was  not  prevalent 
throughout  the  assessment  period.  Although  there  were  some  occurrences  of 
significant  convective  weather,  the  NCWF  may  not  have  had  a  chance  to  be  fully 
utilized. 

5.2  QUESTIONNAIRE  RESULTS. 

Overall  questionnaire  results  for  both  airlines  are  discussed  below.  Individual  airline 
results  are  discussed  in  subsequent  sections. 
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5.2.1  Overall  Dispatcher  Results. 

5.2. 1.1  Utility.  Readability,  and  Accuracy  Ratings. 

Results  of  the  utility,  readability,  and  accuracy  ratings  for  the  NCWF  components  are 
presented  in  table  2.  Note  that  not  all  components  were  rated  on  all  three  dimensions. 
Results  are  shown  using  the  median  and  the  percentage  of  responses  for  each  possible 
rating. 

The  majority  of  NCWF  components  received  an  overall  rating  of  2  (acceptable)  or  above 
on  all  three  dimensions  (utility,  readability,  and  accuracy).  Utility  of  the  airport  identifiers 
and  ARTCC  boundary  overlays  were  rated  highest  overall  with  a  median  score  of  1 
(largely  acceptable). 

Four  of  the  NCWF  components  received  overall  ratings  of  3  (borderline),  indicating  that 
the  component  was  adequate,  but  minor  improvements  would  make  it  more  helpful. 
These  four  were: 

a.  The  1  -hour  forecast  contours  accuracy, 

b.  The  2-hour  forecast  contours  utility, 

c.  The  2-hour  forecast  contours  accuracy,  and 

d.  The  product  update  rate. 

Half  (50  percent)  of  the  responses  indicated  that  the  accuracy  of  the  1 -hour  forecast 
contours  was  borderline.  Although  accuracy  of  the  2-hour  forecast  contours  received 
an  overall  borderline  rating,  over  37  percent  of  respondents  rated  this  component  as 
unacceptable  or  largely  unacceptable.  Thus,  over  1/3  of  the  respondents  indicated  that 
this  component  impedes  their  ability  to  meet  job  requirements  and  leads  to  degradation 
of  job  performance.  While  both  the  1-  and  2-hour  forecast  contours  were  rated  as 
borderline,  the  forecast  direction  lines  and  echo  speed  both  received  overall  ratings  of  2 
(acceptable). 

5.2. 1.2  Job  Task  Ratings. 

Results  of  the  usefulness  of  the  NCWF  for  dispatcher  job  tasks  are  presented  in  table  3. 
Median  rating  scores  were  mostly  positive  with  a  median  score  of  2  (somewhat  useful) 
for  8  of  the  1 1  task  categories.  These  results  indicate  that  the  NCWF  had  a  positive 
effect  on  decisions  and  activities  related  to  the  performance  of  the  following  job  tasks: 

a.  Planning  flight  plans, 

b.  Planning  flight  routes, 

c.  Avoidance  of  severe  convective  weather, 

d.  Selecting  an  alternate  airport, 

e.  Selecting  altitude, 

f.  Monitoring  flight  conditions. 


15 


in 

CO 

CO 

CO 

CO  CO 
ICOiCO  CO 


^  o 


i!  0-2 

^  O  ^ 
O 
03 
c 
D 


CO  p  ^ 


00  h-  o 
cd  CD 

(N  1-  lO 


X-  CO  O) 

r^  ^  cvi 

CO  1“ 


LO  CO  CO 

CO  CD  CD 
CM  CM  CO 


O 

X) 

•I  CO  — 

II  2 
CM  0)  ^ 

O 

O 

< 


■SDirx=5DC<x=>cc< 

Q  ..'...  J,  ..  . 

CO^  (BJ3  o 


col^  u 


TABLE  2.  OVERALL  RATINGS  FOR  NCWF  COMPONENTS  (Continued) 
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TABLE  3.  OVERALL  RATINGS  FOR  NCWF  USEFULNESS  TO  JOB  TASKS 
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g.  Rerouting  flights,  and 

h.  Advising  pilots  enroute  of  severe  convective  weather. 

Three  of  the  task  areas  received  a  borderline  rating  of  3,  indicating  the  NCWF  had  little 
to  no  effect  on  the  performance  of  the  tasks.  These  tasks  were: 

a.  Planning  fuel  quantity, 

b.  Anticipating  ground  delays,  and 

c.  Anticipating  ATC  actions. 

5.2. 1.3  Comments. 

Questionnaire  comments  regarding  improvements  and  product  issues  are  addressed 
under  individual  airline  sections. 

5.2.2  Delta  Airline  Dispatcher  Questionnaire  Results. 

As  mentioned  in  section  5.1,  Delta  Airline  dispatchers  provided  a  limited  number  of 
questionnaire  responses.  In  addition,  it  was  identified  during  interviews  that  even 
among  the  Subject  Matter  Experts,  some  dispatchers  were  using  the  older  1998  version 
of  the  NCWF.  Thus,  much  of  the  Delta  questionnaire  feedback  has  to  be  viewed  as 
inconclusive. 

5.2.2.1  Delta  Airlines  Utility.  Readability,  and  Accuracy  Rating  Results. 

Table  4  displays  the  questionnaire  rating  results  for  Delta  Airlines.  Results  were  not 
generally  consistent  with  overall  questionnaire  results.  Delta  ratings  tended  to  be 
higher,  indicating  a  more  positive  view  of  the  NCWF’s  interfaces  and  components. 
However,  as  noted,  the  small  sample  size  from  Delta  (N=5)  may  or  may  not  be 
considered  representative  of  the  Delta  dispatcher  population. 

For  each  of  the  13  NCWF  components,  median  ratings  were  2  (acceptable)  or  higher  for 
the  three  dimensions  of  utility,  readability,  and  accuracy.  It  was  further  noted  that  60 
percent  (18  of  30)  of  the  component  ratings  were  1  (highly  acceptable),  indicating  that 
most  of  the  NCWF  components  were  viewed  by  the  dispatchers  as  consistently 
enhancing  their  ability  to  meet  job  requirements  or  were  likely  to  lead  to  enhanced  job 
performance. 
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TABLE  4.  DELTA  AIRLINES’  RATINGS  FOR  NCWF  COMPONENTS 
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Also  included  are  the  percentage  responses  of  each  rating  and  the  total  number  of  responses  (N).  Blank  information 
indicates  no  responses  for  the  particular  rating.  See  section  4.3.4.1  for  a  description  of  the  rating  scale. 
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5.2.2.2  Delta  Airlines  Job  Task  Rating  Results. 


As  shown  in  table  5,  Delta  Airline  dispatchers’  overall  rating  scores  were  positive  for  the 
usefulness  of  the  NCWF  in  performing  job  tasks.  Using  the  median  of  the  responses, 
the  NCWF  was  rated  as  1  (highly  useful)  in  the  following  five  areas: 

a.  Preparing  flight  plans, 

b.  Planning  fuel  quantity, 

c.  Avoidance  of  severe  convective  weather, 

d.  Monitoring  flight  conditions,  and 

e.  Rerouting  flights. 

All  other  tasks  received  ratings  of  2  (somewhat  useful)  with  the  exception  of 
"Anticipating  ATC  actions,"  which  received  a  rating  of  3  (borderline). 

5.2.2.3  Delta  Airlines  Comments  and  Enhancements. 

Generally,  Delta  comments  on  the  NCWF  focused  on  the  need  for  an  animation 
capability  (in  order  to  see  trend  information)  and  additional  overlays  such  as  aircraft 
routes  and  navigational  fixes. 

Suggestions  were  given  for  improving  the  readability  of  graphics  and  colors.  A  desire 
for  larger  images,  as  well  as  use  of  different  colors  to  better  delineate  forecast  contour 
lines  were  suggested.  While  these  comments  appear  to  be  inconsistent  with  the 
questionnaire  ratings,  where  readability  was  rated  positively  on  all  components  (see 
section  5.2.2. 1),  the  questionnaire  responses  did  include  several  borderline  ratings  for 
readability,  suggesting  that  minor  improvements  would  make  the  NCWF  more  helpful 
(see  table  4). 

Comments  regarding  the  accuracy  of  the  forecasts  and  storm  top  data  were  provided. 
One  user  reported  that  on  several  occasions,  no  forecasts  were  available  for  VIP  level  3 
or  higher  activity  when  it  was  felt  that  forecasts  should  be  available  (note  that  this  may 
be  a  result  of  the  forecast  criteria  in  section  3.1  not  being  met).  Slow  product  updates 
and  reliability  issues  were  also  identified  (although  problems  with  reliability  were  to  be 
expected  due  to  the  experimental  nature  of  the  NCWF).  The  dispatcher  comments 
appear  to  be  inconsistent  with  the  questionnaire  ratings  on  the  product  update  rate  and 
forecast  accuracy  which  were  rated  1  (largely  acceptable)  or  2  (acceptable). 

Positive  aspects  of  the  NCWF  included  the  usefulness  of  the  forecasts;  the  advantage 
of  the  NCWF  as  a  single  feature  product;  the  graphical  depiction  of  convection;  and 
overall  satisfaction  with  the  day-to-day  performance  of  the  NCWF  during  the 
thunderstorm  season. 

Full  questionnaire  comments  by  Delta  Airline  dispatchers  are  available  in  appendix  F. 
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TABLE  5.  DELTA  AIRLINES’  RATINGS  FOR  NCWF  USEFULNESS  TO  JOB  TASKS 
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5.2.3  Comair  Dispatcher  Questionnaire  Results. 


Approximately  30  dispatchers  from  Comair  provided  responses  to  the  questionnaire. 

The  responses  are  summarized  in  the  following  sections. 

5.2.3. 1  Comair  Utility,  Readability,  and  Accuracy  Rating  Results. 

Comair  questionnaire  rating  results  for  NCWF  utility,  readability,  and  accuracy  are 
shown  in  table  6.  While  results  for  the  NCWF  components  were  not  as  high  as  Delta’s, 
the  majority  of  components  were  still  rated  2  (acceptable)  and  higher.  However,  a 
review  of  the  percentage  responses  indicates  that  a  number  of  Comair  dispatchers  felt 
that  many  of  the  NCWF  components  were  in  need  of  improvement  (i.e.,  a  borderline 
rating).  The  following  items  received  overall  ratings  of  3  (borderline): 

a.  The  1 -hour  forecast  contours  accuracy, 

b.  The  2-hour  forecast  contours  utility, 

c.  The  2-hour  forecast  contours  accuracy, 

d.  The  forecast  direction  lines  accuracy,  and 

e.  The  product  update  rate. 

A  significant  number  of  respondents  (67  percent)  indicated  that  the  1 -hour  forecast 
contours  accuracy  was  either  3  (borderline)  or  4  (unacceptable).  According  to  the  rating 
scale  definitions,  both  borderline  and  unacceptable  are  negative  ratings.  In  addition,  the 
ratings  of  the  2-hour  forecast  contours  should  be  noted.  An  overall  rating  of  3 
(borderline)  for  utility  and  accuracy  was  provided  with  a  substantial  number  of 
respondents  indicating  a  4  (unacceptable)  rating. 

5.2.3.2  Comair  Job  Task  Rating  Results 

Job  task  rating  results  for  Comair  are  shown  in  table  7.  Usefulness  ratings  were  mostly 
positive  with  the  NCWF  receiving  a  median  rating  of  2  (somewhat  useful)  for  7  of  the  1 1 
task  areas.  Unlike  Delta  responses,  no  tasks  received  a  rating  of  1  (of  considerable 
use).  However,  the  seven  task  areas  that  received  the  somewhat  useful  rating  were 
overwhelmingly  positive,  with  over  60  percent  of  the  respondents  rating  the  NCWF  as 
“Of  Considerable  Use”  or  “Somewhat  Useful.”  The  following  three  of  the  seven  task 
areas  had  over  80  percent  of  the  respondents  rating  the  NCWF  positively: 

a.  Planning  flight  route, 

b.  Avoidance  of  severe  convective  weather,  and 

c.  Rerouting  flights. 

The  following  four  task  areas  received  borderline  ratings  (indicating  that  the  NCWF  had 
little  to  no  positive  effect  upon  the  task): 

a.  Planning  fuel  quantity, 

b.  Selecting  altitude. 
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TABLE  6.  COMAIR  RATINGS  FOR  NCWF  COMPONENTS  (Continued) 
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TABLE  7.  COMAIR  RATINGS  FOR  NCWF  USEFULNESS  TO  JOB  TASKS 
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c.  Anticipating  ground  delays,  and, 

d.  Anticipating  ATC  actions. 

The  Planning  Fuel  Quantity  task  was  most  disparately  rated  between  the  two  airlines. 
Delta  users  indicated  the  NCWF  was  of  considerable  use  in  performing  this  task, 
whereas  the  Comair  response  was  borderline.  Interview  information  indicated  that  from 
the  dispatchers’  standpoint,  it  was  very  difficult  to  anticipate  ATC  actions,  regardless  of 
any  product  availability. 

5.2.3.3  Comair  Comments  and  Enhancements 


Comair  questionnaire  comment  summaries  are  located  in  appendix  G.  The  comments 
are  summarized  in  the  following  paragraphs. 

a.  Forecast  Accuracy 

Several  respondents  commented  on  the  NCWF’s  forecast  accuracy,  especially  the 
inaccuracy  of  the  2-hour  forecast.  In  addition,  it  was  noted  that  the  NCWF  performed 
poorly  when  isolated  storm  cells  or  air-mass  conditions  were  present.  Part  of  this 
performance  was,  as  noted  by  some  dispatchers,  due  to  inherent  limitations  in  the 
NCWF  forecast  algorithm.  One  dispatcher  suggested  that  for  air-mass  type  storms, 
other  inputs  (e.g.,  lifted  indices,  outflow  boundaries,  or  winds  aloft)  could  enhance  the 
accuracy  and  overall  usefulness  of  the  NCWF. 

In  addition  to  the  accuracy,  it  was  noted  that  the  1  -  and  2-hour  forecast  contours  tended 
to  vary  a  great  deal  from  one  update  to  another.  As  an  example,  one  dispatcher  related 
an  incident  where  the  NCWF  forecast  a  storm  to  move  east,  then  south,  then  alternated 
between  east  and  south.  In  actuality,  the  storm  moved  southeast. 

Two  dispatchers  indicated  that  the  product  tracked  "phantom"  or  nonexistent  storms. 
ACT-320  personnel  observed  a  similar  situation  and  noted  that  the  forecast  contours 
were  current,  but  the  detection  field  was  outdated.  This  was  brought  to  the  attention  of 
NCAR  personnel  who  believed  it  to  be  a  software  problem. 

These  comments  are  consistent  with  Comair's  questionnaire  ratings,  where  forecast 
accuracy  for  both  the  1-  and  2-hour  forecasts  received  borderline  scores. 

b.  Readability 

The  following  issues  with  readability  were  identified: 

1 .  Annotations  such  as  echo  speed,  storm  tops,  and  airport  identifiers  were 
difficult  to  read  either  because  they  were  too  small  or  surrounding  detection  fields 
obscured  the  annotations.  It  was  suggested  that  larger  print  be  used.  While 
questionnaire  rating  results  indicated  that  25  and  34%  of  the  respondents  gave 
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borderline  ratings  to  the  storm  height  and  echo  speed,  respectively,  overall  readability 
ratings  for  storm  height  and  echo  speed  were  acceptable  (i.e.,  positive). 

2.  The  display  screen  was  too  cluttered,  especially  when  viewing  the  NCWF  on 
the  national  scale  with  all  components  and  overlays  visible. 

3.  The  update  time  needs  to  be  made  more  pronounced  on  the  display. 

4.  The  50-nmi  range  rings  were  squashed  into  ovals.  NCAR  personnel  stated 
that  the  distortion  of  the  circular  range  rings  was  due  to  the  map  projection  used. 

c.  Reliability 

The  NCWF  was  unavailable  on  many  occasions  due  to  system  outages  or  slow, 
inconsistent  updates.  This  was  somewhat  to  be  expected  due  to  the  experimental 
nature  of  the  product  and  due  to  the  NCAR  facilities,  which  are  designed  for  research 
purposes,  rather  than  the  operational  maintainability  of  products.  While  the  product 
reliability  affected  dispatchers’  perceptions  of  the  usefulness  (as  determined  by 
comments),  the  extent  of  the  effect  was  not  measured. 

In  addition  to  reliability,  respondents  reported  that  data  updates  were  too  slow, 
especially  during  dynamic,  convective,  weather  situations.  Based  upon  interview 
comments,  it  is  believed  that  dispatchers  are  referring  to  the  reliability  of  the  NCWF,  not 
that  an  actual  update  rate  faster  than  5  minutes  is  needed,  however,  this  has  not  been 
confirmed. 

The  comments  are  consistent  with  Comair  questionnaire  results  where  the  median 
score  for  update  rate  acceptability  was  borderline. 

d.  Zoom 

Users  reported  problems  with  using  the  zoom  function.  One  noted  difficulties  controlling 
enlargement.  Another  found  that  the  zoom  box  was  difficult  to  see.  One  indicated  that 
re-zooming  an  image  back  to  the  national  map  was  cumbersome  and  a  progressive 
zoom  feature  was  preferred  (although  the  “previous  zoom”  feature  appears  to  already 
give  this  capability,  see  section  3.2.2.1).  These  comments  are  consistent  with 
questionnaire  results  regarding  the  readability  and  utility  of  the  zoom  function.  While 
both  dimensions  received  overall  ratings  of  acceptable,  40  percent  of  users  rated  the 
zoom  utility  as  borderline  and  31  percent  rated  the  zoom  readability  as  borderline  or 
unacceptable. 

e.  Animation 

Over  one-third  of  the  Comair  users  indicated  that  looping  or  animation  would  enhance 
the  utility  of  the  NCWF  in  order  to  see  movement,  growth,  decay,  and  aid  in  predicting 
future  storm  locations.  The  ability  to  animate  appears  very  important  as  one  user  stated 
a  live  radar  loop  with  top  reports  is  just  as,  if  not  more,  useful  than  the  NCWF. 
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f.  Other  Enhancements  and  Comments 


It  was  suggested  that  the  NCWF  should  add  growth  and  decay  components  to  see 
where  and  when  storms  will  build  and  dissipate. 

Two  dispatchers  reported  that  the  NCWF  was  valuable  as  a  supplemental  tool,  but 
could  not  be  relied  on  exclusively. 

One  respondent  noted  the  NCWF’s  greatest  use  was  in  detecting  current  storm 
locations  and  intensity  levels;  yet  also  found  the  1 -hour  forecast  a  useful  tool. 

Various  dispatchers  made  suggestions  for  additional  overlays.  These  included: 

1 .  User  defined  routes  (routes  provided  with  the  NCAR  web  page  were  straight 
lines  and  did  not  reflect  actual  Comair  flight  routes); 

2.  Displayed  flight  routes  to  the  Bahamas,  navaids,  and  user  selected  airports 
and  VORs; 

3.  Wind  directions. 

5.2.4  Delta  Airlines  and  Comair  Questionnaire  Differences. 

Comparisons  of  median  ratings  between  Delta  and  Comair  airlines  regarding 
dimensions  of  utility  and  accuracy  of  the  2-hour  forecast  are  of  special  interest.  While 
Comair  rated  these  dimensions  "borderline,"  Delta’s  median  scores  were  considerably 
higher,  with  an  overall  median  of  1 .5,  indicating  an  acceptable  to  highly  acceptable  view 
on  all  dimensions  of  the  2-hour  forecast. 

Ratings  on  update  rates  between  the  two  airlines  were  also  disparate.  Delta  considered 
the  NCWF  update  rates  largely  acceptable,  whereas  Comair  assigned  a  borderline 
rating  to  this  product  component.  However,  as  already  stated,  Comair  ratings  may  have 
been  more  applicable  to  the  actual  product  reliability  rather  than  the  update  rate. 

In  regards  to  usefulness  to  job  tasks.  Delta’s  responses  tended  to  be  higher  (more 
positive)  than  Comair’s. 

5.3  NONQUESTIONNAIRE  FEEDBACK. 

5.3.1  Comair  User  Log  Comments. 

The  few  comments  provided  via  the  User  Log  were  provided  by  Comair  dispatchers  and 
dealt  mainly  with  the  maneuvering  of  aircraft  in  relationship  to  convective  weather.  It 
was  twice  noted  that  a  decision  was  made  to  not  reroute  aircraft  based  on  forecast 
information  provided  by  the  NCWF.  In  both  cases,  aircraft  were  able  to  stay  on 
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schedule,  rather  than  make  time-consuming  reroutes.  Other  User  Log  information 
included: 

a.  The  existence  of  phantom  cells  (see  section  5.2. 3.3;  Forecast  Accuracy), 

b.  Inconsistent  echo  tops  information, 

c.  No  forecasts  with  thunderstorms  associated  with  Hurricane  Dennis, 

d.  No  forecasts  with  a  slow  moving  line. 

5.3.2  Comair  Observations. 

ACT-320  personnel  observed  Comair  dispatchers  during  normal  operations  to  note  use 
of  the  NCWF.  While  the  observations  were  planned  in  the  anticipation  of  convective 
weather,  no  significant  convection  occurred.  Therefore,  observational  data  was  not 
obtained.  However,  there  were  discussions  with  a  number  of  dispatchers  about  the 
NCWF. 

According  to  dispatcher  comments,  the  NCWF  generally  identified  convection. 
However,  due  to  air  mass  conditions  and  rapid  growth  and  decay,  the  forecast 
component  was  not  always  generated. 

Generally,  the  NCWF  appeared  to  be  an  asset.  Positive  comments  included: 

a.  Easily  accessed, 

b.  Useful  addition  to  existing  products, 

c.  Good  planning  tool, 

d.  Range  rings  useful  for  determining  distances,  and 

e.  Useful  with  lines  of  convection. 

Criticism  of  the  NCWF  was  as  follows: 

a.  Forecast  not  always  displayed, 

b.  Forecasts  were  displayed  without  any  accompanying  detection  field, 

c.  Need  to  be  close  to  display  to  see  graphics  and  text, 

d.  Forecasts  need  to  be  more  accurate.  It  appears  the  NCWF  does  not  assimilate 
enough  of  the  current  storm  situation  and  dynamics. 


Suggested  enhancements  were: 

a.  Animation, 

b.  Extend  geographical  coverage, 

c.  Distance  indicator  or  calculator, 

d.  User  selected  flight  routes, 

e.  Display  lightning  information, 
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f.  Increase  size  of  annotations, 

g.  Include  intensity  in  the  forecast, 

h.  User  selected  cities  and  navigational  aids. 

An  instance  of  beneficial  NCWF  use  was  identified.  A  Comair  pilot  had  requested  an 
additional  passenger  in  lieu  of  fuel  reserves.  Using  the  NCWF,  the  dispatcher  noted  the 
likelihood  of  convection  along  the  route,  which  could  have  lead  to  a  diversion.  Thus, 
fuel  levels  were  maintained. 

5.3.3  Delta  Airlines  Interviews. 


5.3.3.1  On-Site. 

On-site  interviews  were  conducted  with  a  limited  number  of  Delta  Airline  dispatchers. 
Relevant  feedback  consisted  of  the  following: 

a.  Utility: 

1 .  The  NCWF  was  useful  for  planning  and  for  enroute  decisions  concerning  the 
identification  and  status  of  alternate  airports. 

2.  Echo  tops  were  important  to  determine  whether  an  aircraft  could  fly  over  a 
convective  area. 

3.  The  timeliness  of  the  NCWF  was  better  than  existing  operational  products. 

b.  Display: 

1 .  Echo  tops  information  and  airport  identifiers  contributed  to  a  cluttered  display. 
For  the  echo  tops,  it  was  identified  that  perhaps  a  clicking  on  the  displayed  storm  for 
relevant  data  would  be  useful,  although  the  availability  of  the  echo  tops  information 
without  user  intervention  was  good.  It  was  recommended  that  when  an  airport  and  a 
NAVAID  of  the  same  name  were  located  near  each  other,  that  only  one  needed  to  be 
displayed. 

2.  The  time  format  of  hh.mm.ss  was  confusing.  For  example,  18.00.00  was 
interpreted  as  00  Coordinated  Universal  Time  (UTC)  on  the  18th  of  the  month,  rather 
than  18  UTC. 

3.  The  availability  of  more  airports  and  NAVAIDS  was  recommended. 

4.  It  was  recommended  that  NCWF  be  merged  with  Integrated  Terminal 
Weather  System  (ITWS)  and  ASD  information. 

c.  Zoom: 
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1 .  Zoom  function  was  unwieldy,  often  appearing  to  give  a  different  area  than 
what  was  selected. 

2.  A  predetermined  step  function  for  zooming  in  and  out  was  requested,  along 
with  a  pan  feature. 

3.  The  predetermined  ARTCC  regions  of  the  1998  NCWF  version  were 
requested  together  with  the  user-specified  zoom  of  the  1999  version. 

d.  Accuracy: 

1 .  It  was  noted  that  the  NCWF  was  extremely  accurate  for  large,  organized 
storms,  but  not  so  for  air  mass  storms. 

e.  Reliability: 

1 .  The  NCWF  would  often  drop  forecasts,  only  to  have  them  reappear  several 
updates  later.  (Note  that  according  to  the  forecast  criteria,  this  appears  to  indicate  a 
software  problem  rather  than  an  inherent  feature  of  the  NCWF  algorithm.) 

2.  Echo  top  information  was  often  sporadic  in  availability,  which  decreased  its 

utility. 

5.3.3.2  Telephone  Interviews. 

Based  on  telephone  interview  data,  two  instances  of  beneficial  product  use  were 
identified: 

a.  The  NCWF  displayed  a  break  in  what  ATC  personnel  considered  a  solid  line  of 
thunderstorms.  Based  on  the  NCWF  information,  the  dispatcher  was  able  to  route  three 
aircraft  through  the  break. 

b.  The  NCWF  displayed  convection  in  an  area  not  displayed  by  the  WSI  radar 
display.  The  dispatcher  had  his  aircraft  avoid  the  area,  although  other  aircraft 
attempted  to  fly  through.  He  observed  that  these  latter  aircraft  ultimately  had  to  be 
diverted  due  to  thunderstorm  activity. 

In  addition: 

a.  Echo  tops  were  helpful,  for  example,  by  indicating  when  a  storm  was  diminishing. 

b.  Storm  motion  vectors  were  helpful  in  determining  when  storms  would  impact  a 
destination  city  or  a  route. 

c.  The  NCWF  appeared  to  perform  well  on  organized  squall  lines,  but  had  problems 
with  air  mass  thunderstorms  and  Florida  sea-breeze  initiated  storms. 
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d.  NAVAIDs  on  map  tend  to  obscure  views,  especially  with  NAVAIDs  and  airports 
displayed  on  top  of  one  another. 

6.  CONCLUSIONS. 

The  following  sections  summarize  conclusions  in  response  to  individual  assessment 
objectives. 

a.  Objective:  Assess  the  value  of  the  National  Convective  Weather  Forecast 
(NCWF)  in  identifying  and  forecasting  convective  weather  for  airline  dispatch  use. 

Results  indicate  that  the  NCWF  does  provide  value  in  identifying  convective  weather. 
Questionnaire  responses  on  the  utility  of  the  initial  detection  field  were  predominantly 
positive.  Results  concerning  the  forecast  of  convective  weather,  however,  were  mixed. 
The  perceived  value  of  the  1-  and  2-hour  forecasts  differs  significantly  between  the  two 
airline  dispatch  groups.  Comair  rated  the  utility  of  the  2-hour  forecast  as  borderline 
whereas  Delta’s  rating  was  very  positive  (i.e.,  largely  acceptable).  Accuracy  of  the 
1 -hour  forecast,  rated  borderline  by  Comair  personnel,  received  an  acceptable  rating 
from  Delta.  The  borderline  rating  of  the  2-hour  forecast  utility  may  be  correlated  to  the 
perceived  accuracy  of  the  product;  that  is,  less  accuracy  equals  less  utility. 

Feedback  from  open-ended  comments  and  verbal  interactions  provided  additional 
insight  into  the  value  of  the  NCWF.  Dispatchers  indicated  the  NCWF  was  useful  and 
that  the  graphical  representation  of  both  real-time  and  forecast  convection  was  very 
helpful.  However,  Comair  responses  indicated  the  NCWF  was  valuable  as  an  additional 
piece  of  information  rather  than  a  stand-alone  product.  In  addition,  issues  regarding  the 
accuracy  of  the  forecast  components,  especially  the  2-hour  forecast,  lessened  the 
NCWF’s  value  in  forecasting  convective  weather. 

b.  Objective:  Identify  NCWF  benefit  areas  for  dispatcher  tasks  and  decisions. 

Based  on  questionnaire  rating  results,  the  NCWF  appears  to  benefit  most  dispatcher 
tasks  and  decisions.  Overall  results  indicate  the  NCWF  provides  benefit  or  usefulness 
in  the  following  areas: 

1 .  Preparing  flight  plans, 

2.  Planning  flight  routes, 

3.  Avoidance  of  severe  convective  weather, 

4.  Selecting  an  alternate  airport, 

5.  Selecting  altitude, 

6.  Monitoring  flight  conditions, 

7.  Rerouting  flights,  and 

8.  Advising  pilots  enroute  of  severe  convective  weather. 
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Both  airlines  were  consistent  in  rating  the  benefit  of  the  NCWF  to  "anticipating  Air  Traffic 
Control  (ATC)  actions"  as  borderline.  Dispatcher  comments  appear  to  indicate  that 
anticipating  ATC  actions  simply  are  not  possible. 

Comair  rated  more  task  areas  as  borderline  than  Delta.  Most  disparate  among  these 
were  "planning  fuel  quantity."  Whereas,  Delta  found  the  NCWF  "highly  useful,"  Comair 
rated  the  NCWF  as  having  little  or  no  effect  in  performance  of  this  task.  This  disparity 
could  be  a  function  of  the  individual  airlines’  operational  procedures,  type  of  aircraft 
flown,  extent  of  control  over  the  aircraft’s  planned  flight  route,  or  the  small  number  of 
Delta  responses. 

c.  Objective:  Assess  product  graphical  interface  and  characteristics. 

Overall  readability  ratings  from  the  questionnaire  were  acceptable  for  all  components 
with  little  difference  noted  between  the  two  airlines.  While  rating  results  indicated 
acceptability,  comments  indicated  that  some  dispatchers  preferred  a  larger  image  with 
larger  annotations  and  a  more  pronounced  and  clearer  date  and  time  indication. 
Dispatcher  comments  also  included  the  tendency  for  the  NCWF  display  to  become 
cluttered  when  all  components  and  overlays  were  selected.  The  zoom  function  elicited 
several  comments  concerning  the  awkwardness  of  its  use  and  the  desirability  for 
predetermined  steps  and  Air  Route  Traffic  Control  Center  (ARTCC)  regions:  however,  it 
should  be  noted  that  awkwardness  might  be  a  familiarity  or  training  issue. 

d.  Objective:  Assess  NCWF  performance  and  accuracy  from  the  dispatchers’ 
perspective. 

Questionnaire  rating  results  indicated  an  overall  borderline  response  for  the  accuracy  of 
the  1-  and  2-hour  forecast  contours.  From  the  dispatchers’  perspective,  the  accuracy  of 
the  NCWF  forecasts  needs  improvement.  While  the  perspective  of  the  1 -hour  forecast 
was  that  minor  improvements  would  make  it  more  helpful,  a  significant  portion  of  the 
assessment  dispatchers  found  the  2-hour  forecast  unacceptable.  In  contrast, 
questionnaire  results  for  the  direction  lines  and  echo  speed  were  acceptable.  Based 
upon  dispatcher  comments,  the  disparity  between  the  forecasts  and  the  direction  lines 
and  echo  speed  may  be  indicative  of  the  lack  of  convective  growth  and  decay  elements. 
It  may  be  that  the  direction  and  speed  are  correct,  but  changes  in  area  coverage  and/or 
new  growth  are  leading  to  the  borderline  responses  for  the  forecast  contours.  It  was 
noted  that  the  product  appears  to  handle  large,  organized  storms  (especially  line 
storms)  in  an  accurate  manner.  The  performance  with  smaller,  air  mass  storms  is  not 
as  good. 

Other  performance  factors  included  the  lack  of  and  variability  of  forecasts.  This 
appeared  to  be  either  a  result  of  reliability  problems  or  the  inherent  forecast  criteria  of 
the  NCWF.  Forecasts  are  not  produced  unless  storms  meet  certain  size  and  longevity 
criteria.  Dispatchers  noted  that  forecast  contours  could  vary  substantially  from  one 
update  to  another.  This  varying  tended  to  diminish  confidence  in  the  accuracy  of  the 
NCWF. 
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e.  Objective:  Identify  enhancements. 

The  most  frequently  noted  enhancements  are  as  follows: 

1 .  Improved  accuracy  of  the  1  -  and  2-hour  forecasts. 

2.  Animation  to  enhance  the  utility  of  the  NCWF. 

3.  Growth  and  decay  of  convective  areas. 

4.  Larger  graphics  and  text. 

5.  Improved  zoom  capability. 

6.  Additional  overlays,  including: 

(a)  User  defined  flight  routes,  and 

(b)  Additional  Navigational  Aids  (NAVAID). 

f.  Other: 

Poor  reliability  of  the  NCWF  was  a  negative  issue.  Although  reliability  was  not 
addressed  in  the  assessment,  problems  like  down-time  and  presentations  of  old  data 
may  have  influenced  participants’  perception  of  the  product.  The  borderline  rating  given 
to  the  product  update  rate  may  be  more  of  an  indication  of  product  reliability.  This  was 
supported  by  comment  information  and  interviews  where  dispatchers  noted  that  the 
product  was  not  always  available.  Since  the  NCWF  was  an  experimental  product,  its 
availability  could  not  be  ensured  on  a  24-hour  per  day,  7-day  per  week  basis.  Users 
tended  to  interpret  unavailability  as  an  inherent  problem  of  the  product  itself.  However, 
it  should  be  noted  that  written  comments  identified  product  update  issues  to  the  extent 
that  data  updates  are  too  slow  and  therefore  not  synchronized  with  real-time  conditions. 
This  is  especially  important  when  considering  rapidly  changing  conditions  inherent  with 
some  convective  weather  situations. 

Participation  from  Delta  Airlines  was  disappointing.  Although  the  participating 
assessment  group  was  small,  commitment  from  each  participant  and  management  was 
assumed  based  upon  coordination  both  before  and  during  the  assessment.  However, 
there  were  no  user  log  responses  and  only  five  completed  questionnaires.  The  lack  of 
involvement  causes  uncertainty  with  respect  to  how  representative  the  Delta  data  was. 

7.  RECOMMENDATIONS. 

a.  It  appears  the  National  Convective  Weather  Forecast  (NCWF)  product  concept  is 
useful  for  airline  dispatcher  use.  The  product  interface  appears  to  be  acceptable, 
although  a  few  modifications  could  be  performed. 

b.  The  strongest  criticism  of  the  NCWF  is  in  regards  to  its  accuracy  of  the  1-  and  2- 
hour  forecasts.  Further  development  should  concentrate  on  improved  accuracy  of  the 
forecasts,  including  the  incorporation  of  growth  and  decay  capabilities. 
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c.  The  assessment  provided  support  for  the  utility  of  the  NCWF  product  concept  to 
airline  dispatch  operations.  Further  work  should  be  performed  with  Air  T raffic  Control 
(ATC)  facilities  in  order  to  provide  a  similar  convective  forecasting  capability,  enhancing 
shared  situation  awareness  between  dispatch  and  controller  functions. 


8.  ACRONYMS. 

ACSF 

Automated  Convective  SIGMET  Forecast 

ACT-320 

Weather  Branch  at  the  FAA  Technical  Center 

ARTCC 

Air  Route  Traffic  Control  Center 

ASD 

Aircraft  Situation  Display 

ATC 

Air  Traffic  Control 

AWC 

Aviation  Weather  Center 

AWRP 

Aviation  Weather  Research  Program 

dB 

decibels 

FAA 

Federal  Aviation  Administration 

ITWS 

Integrated  Terminal  Weather  System 

km 

kilometers 

NAVAID 

Navigational  Aid 

NCAR 

National  Center  for  Atmospheric  Research 

NCWF 

National  Convective  Weather  Forecast 

NLDN 

National  Lightning  Detection  Network 

nmi 

nautical  miles 

NOAA 

National  Oceanic  and  Atmospheric  Administration 

PIREP 

Pilot  Report 

RAP 

Research  Applications  Program 

SIGMET 

Significant  Meteorological  Statements 

TITAN 

Thunderstorm  Identification  Tracking  and  Nowcasting 

UTC 

Coordinated  Universal  Time 

VHF 

Very  High  Frequency 

VIL 

Vertically  Integrated  Liquid 

VIP 

Video  Integrated  and  Processor 

VOR 

VHF  OmniRange  Navigation  System 

WSI 

Weather  Services  International 

WSR-88D 

Weather  Surveillance  Radar  -  1988  Doppler 
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APPENDIX  A 


COMAIR  BASELINE  INFORMATION 


1.  Background: 

In  support  of  the  NCWF  1999  assessment,  baseline  data  collection  was  conducted  at  Comair 
Airlines  from  June  1  -  2, 1999  at  airline  headquarters  at  the  Cincinnati  International  Airport, 
Covington,  Kentucky.  Eight  dispatchers  were  interviewed.  In  addition,  a  Lead  Dispatcher  and 
the  Duty  Manager,  Flight  Operations  gave  an  overview  of  Comair  operations,  which  is 
summarized  below. 

Comair  is  the  largest  regional  airline  in  the  USA  and  a  regional  carrier  for  Delta  Airlines.  It 
operates  nearly  700  daily  flights.  The  airline  is  headquartered  at  the  Cincinnati/Northern 
Kentucky  International  Airport.  Comair  has  flights  between  80  cities  in  28  states  and  3  countries 
(Nassau,  the  Bahamas,  and  Canada).  Flight  routes  encompass  Montreal,  Canada  to  the  North; 
Key  West  and  Nassau  to  the  South;  Bangor,  Maine  to  the  East;  and  Colorado  Springs  to  the 
West. 

The  airline,  to  date  operates  2  aircraft  types:  1)  Canadair  Regional  Jet  (CRJ)  and  2)  Embraer 
Brasilia  (EM2).  The  CRJ  is  a  50  passenger  turbofan  aircraft.  To  date  76  CRJs  are  in  operation. 
Comair  intends  to  buy  50  additional  CRJs  over  the  next  year  with  future  options  to  obtain  1 1 5 
more.  The  airline  plans  to  phase  out  the  smaller  turboprop  EM2s  that  carry  up  to  30 
passengers.  Currently  there  are  25  EM2s  operating. 

2.  Organization  of  Aircraft  Operations  Center: 

Dispatchers  are  located  in  the  Systems  Operations  Control  Center  (SOCC).  The  SOCC 
consists  of  Operations  and  Technical  Support.  Dispatch  falls  under  Operations  and  shares  the 
SOCC  area  with  crew  schedulers,  maintenance  coordinators,  flight  controllers,  and  flight 
followers. 

In  the  dispatch  area  there  are  four  duty  managers,  two  lead  dispatchers  and  approximately  eight 
regular  dispatchers.  Dispatchers  are  not  assigned  flights  based  on  aircraft  type  or  geographical 
location,  like  other  airlines,  but  by  lines  of  flight.  These  schedules  appear  to  be  arbitrarily 
computer  generated  and  are  rotated  from  day  to  day.  Typically,  the  dispatcher  will  not  know 
until  the  day  before  which  lines  he/she  will  be  controlling.  The  exception  to  this  is  the  lead 
dispatcher  who  has  taken  over  the  newly  created  shuttle  desk.  He/she  will  handle  the  same 
flights  daily.  Generally  each  individual  dispatcher  handles  40  to  50  flights  a  day.  Shifts  are 
broken  out  as  follows:  4  am  -  2  pm;  5  am  -  3  pm;  6:30  am  -  4:30  pm;  8:45  am  -  5:45  pm;  1:30 
pm  - 1 1 :30  pm;  2:45  pm  -  12:45  am;  and  6:30  pm  -  4:30  am.  Operations  typically  end  at  1 :00 
am  and  resume  at  5:30  am.  There  are  a  few  overnight  charters. 

There  is  one  ATC  liaison  known  as  the  flight  controller.  The  flight  controller  communicates  with 
ATC  centers,  TMUs  and  the  Command  Center.  There  are  scheduled  telecons  with  the 
Command  Center  and  Centers  at  8:15  am  and  12:30  pm.  Generally  issues  like  weather 
conditions,  booking  levels  and  VIP  operations  are  discussed. 

The  dispatcher  interfaces  with  a  number  of  individuals  who  are  responsible  for  various  aspects 
of  the  flight  and  for  providing  important  information.  For  example,  the  scheduling  coordinator 
will  inform  the  dispatcher  of  changes  in  crews  at  least  2  hours  In  advance.  The  maintenance 
planner  provides  information  on  restrictions  on  the  MEL  (minimum  equipment  list),  any 
equipment  problems,  and  what  aircraft  have  undergone  maintenance.  The  maintenance 
coordinator,  located  next  to  the  flight  controller,  also  passes  on  maintenance  information  such 
as  aircraft  maintenance  status.  The  flight  controller  or  supervisor  oversees  all  dispatch 
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activities  and  provides  information  such  as  cancelled  flights,  crew  changes,  aircraft  changes, 
and  late  arriving  flights.  Flight  followers  generally  monitor  all  flights  and  pass  on  status 
changes,  flow  control  messages,  or  other  information  that  may  be  related  to  a  flight. 

3.  Dispatch  Workspace  Configuration: 

Resources  (hardware  and  software)  available  to  dispatchers  and  other  dispatch  personnel 
include  the  following: 

Deltamatic  -  Standalone  display  with  flight  follower  application  providing  flight  information, 
flight  times,  payloads  messages,  etc.  This  is  available  to  each  dispatcher. 

X:  Terminal  -  PC  providing  applications  such  as  Kavouras  radar,  flight  release  software  and 
Netscape  Internet  browser.  Dispatchers  can  also  receive  e:mail  messages,  over  the  airline’s 
intranet.  Most  internal  information/messages  are  disseminated  in  this  manner.  This  is  available 
to  each  dispatcher. 

Air  to  ground  radio  -  This  is  used  to  communicate  to  flights  enroute  that  are  in  range  - 
range  extends  from  Cincinnati  to  Orlando,  FL,  Beyond  Orlando  (Comair’s  2^  hub)  enroute 
radio  communication  is  patched  via  telephone.  This  is  available  to  each  dispatcher. 

ASD  -  Aircraft  Situation  Display  is  used  to  graphically  follow  flights;  especially  those  that  are 
not  followed  on  the  Deltamatic.  The  application  can  provide  status  on  all  flights  and  is  useful  in 
determining  how  ATC  is  routing  traffic.  Weather  radar  overlays,  locators,  and  distance 
calculators  make  this  a  very  useful  resource.  However,  there  is  only  one  ASD  located  on  the  far 
side  of  the  room,  near  the  lead  dispatcher  area. 

4.  Preparing  Flight  Plans: 

The  flight  plan  is  automatically  generated  via  the  WXAIR  system.  The  dispatcher  only  needs  to 
enter  the  captain’s  name  and  aircraft  number.  The  system  will  then  generate  the  preferred 
route,  contingency  fuel  levels,  weight  restrictions,  preferred  speed  (via  forecast  winds),  and 
current  and  forecast  weather  both  enroute  and  at  the  destination  city.  Weather  parameters  can 
be  integrated  into  the  program  in  terms  of  intensity  and  types  of  weather.  This  cuts  down  on  the 
amount  of  text  the  pilot  must  read  -  filtering  out  the  less  critical  or  superfluous  information. 
Weather  sources  include  SIGMETs,  AIRMETs,  forecast  winds,  and  advisories.  PIREPs  can  be 
added  if  desired.  The  dispatcher  chooses  an  alternate  airport  (if  required).  The  flight  plan  is 
prepared  2  hours  prior  to  flight  and  issued  1  hour  in  advance.  If  situations  change  (e.g., 
changes  in  weather  along  route)  the  flight  plan  can  be  amended.  If  enroute,  the  dispatcher  can 
radio  the  pilot  and  verbally  advise  of  the  change.  If  the  aircraft  is  still  on  the  ground,  the 
dispatcher  can  call  the  station.  If  necessary,  messages  to  the  pilot  can  be  left  at  the  gate  using 
the  RIDS  application. 

A  summary  of  the  flight  plan  is  sent  to  ATC’s  data  center.  The  flight  crew  will  call  ATC  if  the 
flight  plan  changes. 

Workload  is  heaviest  during  the  “pushes”.  There  are  4  pushes:  from  8  am  -  8:40  am;  10  am  - 
10:40  am;  12  pm  -  12:40  pm  and  2  pm  -  2:40  pm.  The  8am  and  2  pm  pushes  include  the 
highest  number  of  aircraft  -  thus  the  heaviest  dispatch  workload. 

5.  Enroute  Monitoring: 

Kavouras  radar  is  used  to  monitor  flight  routes.  If  weather  is  severe,  traffic  management  (ATC) 
can  be  called  to  discuss  flight  route  options  -  available  openings  in  the  weather.  PIREPS  are 
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also  referenced  and  pilots  are  encouraged  to  call  in.  Also,  PIREPs  listed  on  the  Deltamatic  are 
accessed.  These  are  updated  every  hour.  Indianapolis  and  Chicago  enroute  Centers  handle 
75  -  80%  of  Comair  flights. 

Only  flights  to  and  from  Colorado  Springs  and  Nassau  require  pilots  to  call  in.  Generally 
weather  is  monitored  for  enroute  flights  and  messages  are  exchanged  if  weather  conditions 
change  significantly.  Pilots  are  required  to  call  dispatch  if  there  has  been  an  ATC  reroute. 
Under  this  circumstance,  the  dispatcher  will  issue  another  plan  with  new  calculations  for  optimal 
fuel  burn.  When  flight  plans  change,  the  dispatcher  must  have  other  optional  plans  on  hand. 

6.  Weather: 

Generally,  weather  information  sources  include  the  following: 

a.  Kavouras  radar  composites  with  30  -  40  minute  animations, 

b.  Displays  of  individual  NEXRADs, 

c.  Convective  SIGMETS  (textual), 

d.  PIREPs, 

e.  Tower  information  regarding  ground  fog, 

f.  Weather  Channel  (not  an  official  source). 

In  addition  to  the  above  information,  dispatchers  were  interviewed  and  asked  about  weather 
sources  used;  how  flight  planning  and  monitoring  tasks  are  performed;  and  requirements  they 
would  have  for  an  automated  convective  detection  and  forecast  product.  The  following  sections 
contain  a  summary  of  their  responses. 
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7.  Weather  Sources: 


a.  What  weather  sources  are  available  to  you? 

b.  Which  do  you  use  most  -  or  find  most  expedient  or  reliable? 

c.  What  is  your  primary  source(s)  for  convective  weather  information?  Why? 


Title 

Response 

Dispatcher  1 

1)  SIGMETS;  2)  AIRMETs;  3)  Briefing  from  previous  dispatcher;  4)  Self-brief 
by  looking  at  the  available  weather  information  ;  5)  300-250  mb  charts 
displaying  temperature,  icing  and  winds  aloft;  6)  Area  Forecasts  - 12  to  24 
hours;  7)  METARS  and  TAPS  throughout  the  shift;  8)  Composite  radar  (used 
mostly  for  routing  -  to  see  if  there’s  any  gaps  in  the  weather);  9)  NEXRAD; 

10)  Jet  stream  maps  (mostly  in  winter);  and  10)  PIREPS  -  PlREPs  are 
especially  useful  for  icing 

Dispatcher  2 

1)  Radar  mostly  -  both  composite  and  NEXRAD.  2)  SIGMETs  which  are 
mentally  plotted;  and  3)  PlREPs  (sometimes). 

Dispatcher  3 

1)  Radar  composites  to  see  how  fast  the  weather  is  moving;  2)  SIGMETs  (to 
see  speed  and  the  direction  of  movement);  3)  Radar  reports  (being  phased 
out  by  NWS);  4)  Maximum  tops  (very  important  information);  5)  NEXRAD  (to 
look  for  holes  and  see  how  close  -  cannot  calculate  distances  now  -  mostly 
guess  work);  6)  METARS;  7)  TAFs,  and  8)  PIREPS  (occasionally,  especially  if 
they  are  from  Comair  crews). 

Dispatcher  4 

1 )  Kavouras  radar  (regional  display);  2)  Current  radar  with  infrared  satellite 
overlay.  Has  difficulty  discerning  radar  colors  in  terms  of  high  intensity 
(browns,  reds,  et  al)  especially  between  levels  3  and  4;  and  3)  METARs. 
Reported  problems  switching  between  different  weather  information  sources  - 
too  timely.  Needs  to  know  how  weather  will  develop.  Uses  individual 
NEXRADs  on  approaches  in  winter,  however  prefers  regional  radar  views. 

Dispatcher  5 
(Lead) 

1)  Kavouras  composite  radar  with  airports  and  VORs  overlaid;  2)  NEXRAD  for 
selected  cities  (though  not  good  for  broader  picture);  3)  Infrared  satellite  (to 
make  decisions  on  finals  and  whether  to  head  east  or  west);  4)  PIREPS  over 
the  route  of  flight  (from  Jeppeson  system);  and  5)  ASD  (for  shared  situational 
awareness  -  can  see  if  other  planes  have  flown  through  a  questionable  route 
and  then  confirm  with  the  TMU  at  the  corresponding  Center). 

Dispatcher  6 

1)  Animated  radar  composite  -  from  the  past  hour;  2)  NEXRAD  (to  check 
stations  and  get  a  closer  focus  -  also  to  see  holes  in  weather);  3)  Convective 
SIGMETs  (to  see  how  fast  the  weather  is  moving  and  get  tops,  speed  and 
mileage  rings);  and  4)  PlREPs  (for  tops  and  movement  -  also  can  get 
distances  from  the  pilots). 

Dispatcher  7 

1)  Convective  SIGMETs  (for  direction,  speed,  tops);  compare  to  Kavouras 
radar  and  try  to  guess  where  the  weather  will  be  in  2  hours  although  cannot 
account  for  increase  or  decrease  in  cell  intensity  or  when  cells  will  dissipate; 

2)  NEXRAD  radar  for  comparison  of  PlREPs  along  the  flight  route,  especially 
if  they’re  going  through  an  area  of  thunderstorms-  and  to  get  cell  tops. 

Dispatcher  8 

1 )  Surface  analysis  charts  (for  fronts  -  obtained  via  Kavousas  -  not  updated 
enough);  2)  Composite  radar  (to  see  where  fronts  have  moved);  3)  300  mb 
chart  (with  jet  stream,  winds,  turbulence);  4)  Weather  depiction  chart;  4) 
Release  sheet  (monitor  where  flight  is  and  narrow  down  the  view);  5) 

NEXRAD  for  Cincinnati;  6)  Non-graphical  SIGMETs  (draws  mental  picture);  7) 
Individual  city  forecasts;  8)  METARs;  9)  TAFs;  and  10)  PIREPS. 
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8.  Flight  Planning: 


a.  Describe  typical  activities  in  preparing  a  flight  plan. 

b.  Who  do  you  interface  with?  How? 

c.  What  weather  information  sources  do  you  use  (if  different  from  above)?  Why? 


Titie 

Response 

Dispatcher  1 

1 )  First  look  at  aircraft  -  passenger  and  cargo  loads.  2)  Checks  weather  to 
determine  the  route  of  flight,  alternates,  weights  and  the  ability  to  carry  extra 
fuel.  3)  Checks  CSLs  and  MELs  for  the  performance  of  the  aircraft  to  see  if 
altitude  drop  is  necessary  -  and  will  check  again  to  see  if  weights  are  within 
weight  criteria.  4)  Checks  in  on  flight  to  get  the  most  recent  information  from 
the  airport.  5)  If  necessary,  modifies  weather  reports  based  on  TAFs,  NOTAMS 
and  PIREPs. 

Dispatcher  2 

1 )  Makes  sure  that  the  previous  flight  has  arrived  on  time  2)  Looks  at  fueling 
requirements  3)  Checks  the  weather  (most  important  activity,  especially  if 
reroutes  are  required). 

Dispatcher  3 

1 )  Checks  routes  to  see  where  flights  will  be  2)  Checks  weather  by  looking  at: 
METARs;  TAFs;  PIREPs;  Winds  Aloft;  SIGMETs;  CWAs;  to  see  how  the 
weather’s  moving  and  how  to  get  around  it. 

Dispatcher  4 

Generally  given  12  to  13  lines  of  flights.  1)  Receives  briefing  from  previous 
dispatchers;  2)  Tries  to  get  ahead  on  releases  -  prepare  them  ahead  of  time  - 
so  that  there  is  more  time  for  the  unexpected,  (needs  to  know  what’s  going  to 
happen  in  the  next  2-3  hours);  3)  looks  at  weather  on  the  radar;  4)  Checks 
MELS;  5)  Automatic  download  of  flight  plan  one  hour  before  the  flight  departs; 
6)  Continues  to  monitor  weather. 

Dispatcher  5 
(Lead) 

1 )  Looks  at  where  flights  are  arriving  and  departing  on  an  ongoing  basis;  2) 
Checks  radar  for  current  weather  and  calculates  payload  and  booking  effects  - 
with  short  hauls  between  Boston  and  NYC  has  to  handle  Vz  the  lines  of  other 
dispatchers.  Shuttles  use  CRJs  -  Brasilias  and  are  used  for  1  hr.  hauls  - 
mostly  in  Florida. 

Dispatchers 

1 )  Looks  at  weather  -  AIRMETs,  SIGMETs,  outlooks;  graphical  weather  (12 
hour  forecast  and  prognosis);  2)  Checks  MELs  that  system  brings  up;  3)  Issues 
flight  plan  1  hour  prior  to  departure. 

Dispatcher  7 

1 )  Verify  crew  names  on  X  windows;  2)  Checks  Kavouras  radar;  3)  Looks  at 
WXAIR  system  for  TAFs,  METARs,  etc.;  4)  Uses  Deltamatic  as  a  reference  for 
passenger  loads  and  if  the  flight  is  on  time;  5)  Checks  satellite  imagery  for  a 
better  weather  view. 

Dispatcher  8 

1 )  Checks  destination  weather;  2)  Checks  passenger  loads;  3)  Checks  if 
“round-tripping”  is  feasible;  4)  With  worsening  weather  conditions,  will 
communicate  to  station  personnel;  5)  Discuss  alternate  routes  with  the  pilot. 

A-5 


9.  Flight  Replanning; 


a.  Describe  typical  activities  in  replanning  a  flight  plan  once  an  aircraft  is  enroute  due  to 
weather  avoidance,  mechanical  problems,  traffic  delays,  etc. 

b.  Who  do  you  interface  with?  How? 

c.  What  are  the  criteria  for  changing  plans,  e.g.,  how  bad  does  the  weather  have  to  be? 

d.  How  specifically  does  severe/significant  convective  weather  along  a  flight  route  affect 
flight  replanning? 


Title 

Response 

Dispatcher  1 

1 )  Check  what  has  changed  -  looks  at  Deltamatic,  maps,  and  routes  - 
generally  relies  on  radar;  2)  Tries  to  vector  aircraft  around  where  storm  will  be; 

3)  Calculates  intersections  (rerouting  program  doesn’t  calculate  intersections); 

4)  Reads  new  fuel  burns  over  the  phone  to  the  pilot;  5)  Makes  amendments  to 
the  original  release;  6)  Makes  note  on  release  for  the  amendment. 

Dispatcher  2 

1)  Safety  of  flight  is  #  1  concern;  2)  Sees  how  ATC  is  routing;  3)  recalculates 
and  issues  new  fuel  burn  numbers;  4)  Checks  radar  for  weather  conditions;  5) 
Briefs  pilot  -  ATC  route  may  not  be  good  -  may  ask  pilot  to  request  another 
route;  6)  Looks  at  SIGMETs  to  see  how  to  calculate  fuel  burn;  7)  Continues  to 
check  radar  with  animation. 

Dispatcher  3 

1 )  Call  crews  if  enroute;  2)  Checks  radar;  3)  Coordinates  with  flight  crew  -  asks 
them  to  check  their  radar  and  help  make  a  decision;  4)  Look  for  reasonable 
alternate  route;  5)  See  if  ATC  will  allow  another  route;  6)  Make  fuel  burn 
recalculations. 

Dispatcher  4 

1)  Receives  call  from  the  pilot  if  radar  is  showing  red;  2)  Most  concerned  with 
Brasilias,  since  they  cannot  carry  much  fuel;  3)  Confers  with  pilots  on  potential 
reroutes;  4)  Can  redispatch  flight  or  pull  up  old  release  and  replan. 

Dispatcher  5 
(Lead) 

1 )  If  conditions  worsen  30  minutes  to  1  hour  before  the  flight,  can  contact  the 
pilot  on  the  ground;  2)  Calls  to  say  there  is  a  new  SIGMET  and  advises  of 
possible  reroute;  3)  If  ATC  puts  in  new  program,  then  reroutes  are  decided;  4) 
New  fuel  burn  is  calculated;  5)  Check  weather  radar  for  clear  air;  6)  Will 
sometimes  confer  with  pilot. 

Dispatcher  6 

1)  Uses  Deltamatic  reroute  screen;  2)  Confers  with  flight  crew. 

Dispatcher  7 

1 )  If  anticipated,  will  load  extra  fuel;  2)  Plans  reroutes;  3)  ATC  may  be  holding 
even  if  weather  is  good,  i.e.  traffic  saturation. 

Dispatcher  8 

1 )  Prepare  reroutes  -  uses  redispatch  screen  to  come  up  with  new  routes;  2) 

Use  WXAIR  system  to  check  weather  conditions;  3)  Confers  with  pilot;  4)  Calls 
center  TMUs. 
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10.  Flight  Monitoring: 


a.  Describe  typical  activities  while  monitoring  flights,  e.g.,  assessing  weather  conditions 
enroute  and  at  destination  airports? 

b.  Who  do  you  interface  with?  How?  When? 

c.  What  information  sources  (i.e.,  weather)  do  you  use?  Why? 


Titie 

Response 

Dispatcher  1 

1 )  Checks  ASD  display;  2)  Check  in  and  out  times  at  gates  to  see  if  late  or  on 
time;  3)  With  appearance  of  heavy  convective  weather  will  a)  monitor  ASD;  b) 
try  to  see  where  ATC  is  routing  other  flights;  c)  see  where  flights  are  lined  up; 
d)  contact  flight  crew  on  Delta’s  radio;  e)  issue  bulletins;  f)  has  flight  follower 
disperse  information  to  other  dispatchers. 

Dispatcher  2 

If  good  day,  no  need  to  do  much  monitoring.  If  weather  is  bad,  will  monitor 
SIGMETs.  May  look  at  the  ASD  to  see  if  other  aircraft  are  getting  through. 

Ability  to  click  weather  on  and  off  on  the  ASD  display  is  very  helpful. 

Dispatcher  3 

If  there  is  weather,  will  look  at  the  Deltamatic  to  check  on  flights  -  see  if  they’ve 
arrived  on  time.  Will  call  the  crew  when  unexpected  weather  or  turbulence 
appears. 

Dispatcher  4 

1 )  Checks  ASD  often  -  sometimes  there  are  gaps  but  it  is  usually  accurate;  2) 
Overlays  weather;  3)  Overlays  other  flights;  4)  Relays  information  to  pilots  - 
especially  if  there  are  areas  (holes)  to  fly  through;  5)  Uses  Deltamatic  flight 
follower  in  conjunction  with  ASD  and  checks  departure  and  arrival  times;  6) 
Looks  at  TOAST  options  for  additional  information;  7)  Uses  air  to  ground  radio 
to  contact  flight  crew. 

Dispatcher  5 
(Lead) 

1 )  When  monitoring  during  bad  weather  wants  to  know  where  aircraft  are;  2) 
checks  flights  in  relation  to  weather;  3)  Looks  at  ASD  display;  3)  Uses 

Deltamatic  as  the  primary  information  source  for  enroute  monitoring;  4) 

Requires  crew  to  call  in  if  encountering  bad  weather. 

Dispatcher  6 

1 )  Monitors  closely  when  weather  is  bad;  2)  Checks  Deltamatic  to  see  if  flights 
arrived  and  departed  on  schedule;  3)  Looks  at  ASD;  4)  Rechecks  weather  and 
fuel  loads. 

Dispatcher? 

1 )  Watches  each  flight;  2)  Checks  on  arrival/departure  times  -  if  flight  is  30 
minutes  late,  will  note  on  the  flow  sheet;  3)  Looks  at  Deltamatic;  4)  Checks 

ASD  and  determines  why  flights  are  holding. 

Dispatcher  8 

1)  Tracks  each  flight  using  a  flow  sheet;  2)  Checks  arrival/departure  times  on 
Deltamatic;  3)  Uses  TOAST  system;  4)  Checks  ASD. 
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11.  What  would  you  like  to  see  in  a  convective  detection  and  forecast  product? 


Titie 

Response 

Dispatcher  1 

1)  Ability  to  see  holes  and  gaps  in  the  weather;  2)  Mileage  key  to  calculate 
distances;  3)  Individual  storm  cell  vectors;  4)  Storm  Vectors;  5)  Navaids 
overlay  (user  selectable). 

Dispatcher  2 

1)  Tops;  2)  Storm  movement;  3)  Strength  (intensity)  of  storm. 

Dispatcher  3 

1)  Tops;  2)  Forecast  -  at  least  1  hour;  3)  Storm  movement;  4)  Overlays  of 

VORs,  jet  routes,  and  arrival  and  departure  gates 

Dispatcher  4 

1)  Overlay  VORs  and  significant  cities;  2)  Overlay  product  on  ASD; 

Dispatcher  5 
(Lead) 

1 )  Cloud  tops  -  may  aid  in  understanding  the  presence  of  turbulence;  2) 

Doppler  radar  -  to  see  horizontal  wind  shear;  3)  Height  of  cloud  layers;  4) 
Lightning  data. 

Dispatcher  6 

1)  Overlay  VORs;  2)  Show  6  levels  of  weather;  3)  Same  look  as  composite 
radar;  4)  Tops;  5)  Animation;  6)  Storm  movement. 

Dispatcher  7 

1 )  Weather  briefing  from  meteorologists;  2)  Growth  and  decay;  3)  Overlays  of 
VORs  and  jet  routes. 

Dispatchers 

1)  Forecast  of  1  to  1  Va  hours;  2)  Storm  velocity;  3)  Tops;  4)  Flight  routes 
overlaid  with  radar;  5)  Capability  to  overlay  weather  charts. 

Dispatcher  9 
(Lead) 

1)  Heights  of  cloud  layers  ;  2)  Plotted  Convective  SIGMETs;  3)  Cloud  tops;  4) 
Geographical  locators;  and  5)  Distance  calculators. 
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APPENDIX  B 


DELTA  AIRLINES  USER  LOG 


NATIONAL  CONVECTIVE  WEATHER  FORECAST  (NCWF) 
MONITORING  LOG  FOR  DELTA  AIRLINES’  DISPATCHERS 


Instructions:  Please  take  a  few  minutes  to  write  down  your  impressions  of  the  NCWF.  The  log 
has  been  broken  out  into  four  general  sub-areas  for  your  consideration:  1)  Perceived  Accuracy; 
2)  Ease  of  Use;  3)  Effect  on  Dispatch  Task(s);  and  4)  Additional  Comments.  Please  be  specific 
regarding:  geographical  locations;  flight  routes  effected;  weather  phenomena  encountered;  and 
types  and  sources  of  information  used. 

Responses  to  this  log  will  remain  anonymous  and  confidential.  No  individual  will  be  associated 
with  any  comments. 


PLEASE  PROVIDE  THE  FOLLOWING  INFORMATION. 

Name  or  identifier:  _  Title: _ 

Type  of  haul  (check  one):  Short  Medium  Long  Other 

PERCEIVED  ACCURACY: 


EASE  OF  USE: 


EFFECT  ON  DISPATCH  TASK(S): 


ADDITIONAL  COMMENTS: 


APPENDIX  C 


COMAIR  USER  LOG 
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♦CONVECTIVE  WX  TYPE  =  LINE  STORMS,  AIR  MASS,  STATIC,  RAPIDLY  CHANGING 


APPENDIX  D 


DELTA  AIRLINE  TELEPHONE  INTERVIEW  QUESTIONS 


Telephone  Interview  Questions 


a.  It  appeared  that  convective  activity  occurred  from  (time  of  onset  -  time  of  offset).  Is  that 
true?  Were  you  anticipating  it? 

b.  How  did  this  weather  affect  your  flights? 

c.  During  this  time,  how  did  you  use  the  NCWF  or  think  it  could  be  used/applied? 

d.  How  did  the  NCWF  perform  for  you?  Can  you  recall  examples  of  that  performance? 

1)  If  good  or  bad,  how  did  you  know?  For  example,  did  you  compare  it  to  pilot  reports 
(PIREPS)  orASDdata? 

2)  Did  this  performance  affect  your  confidence  in  the  accuracy  of  the  product?  If  so,  how 
was  your  confidence  affected? 

3)  In  this  situation,  which  product  components  did  you  find  most  useful?  Why? 

e.  How  did  use  of  the  product  impact  your  tasking  or  dispatch  activities? 

f.  What  problems  did  you  encounter  (e.g.,  reading  text;  obscuration  of  data;  zooming; 
navigation;  differentiating  colors  and  intensity  levels;  updates  too  fast  or  slow)? 

g.  Is  there  anything  else  you  would  like  to  add? 
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APPENDIX  E 


DISPATCHER  QUESTIONNAIRE 


1999  NATIONAL  CONVECTIVE 
WEATHER  FORECAST  (NCWF) 
ASSESSMENT 

DISPATCHER 

QUESTIONNAIRE 


Prepared  by: 

Communication/Navigation/Surveillance 
Engineering  and  Test  Division,  Weather  Branch,  ACT-320 
William  J.  Hughes  Technical  Center 
Federal  Aviation  Administration 
Atlantic  City  International  Airport 
Atlantic  City,  NJ  08405 
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Please  provide  the  following  information: 

Airline: _  Title: _ 

Delta  Participants:  Please  check  or  indicate  the  dispatch  area  you  cover 

Short  Haul _  Medium  Haul _  Long  Haul _  Other 


Instructions 


The  purpose  of  this  questionnaire  is  to  obtain  feedback  from  airline  dispatchers  regarding  the 
National  Convective  Weather  Forecast  (NCWF)  product.  The  Aviation  Weather  Research 
Branch  at  the  FAA’s  William  J.  Hughes  Technical  Center  is  conducting  the  assessment  of  the 
NCWF  product. 

Feedback  from  dispatchers  is  a  very  important  component  of  the  assessment,  and  responses  to 
this  questionnaire  will  provide  important  information  for  use  in  future  iterations  of  the  NCWF 
product.  Please  respond  to  all  questions  as  honestly  and  thoroughly  as  possible. 

All  responses  will  remain  ANONYMOUS  and  CONFIDENTIAL.  A  report  will  be  written  on  the 
results  of  this  questionnaire,  however,  no  one  will  be  identified  or  associated  with  any  specific 
comment.  Please  return  the  questionnaire  to  the  FAA’s  Technical  Center  Evaluator  or 
designated  representative. 
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Rating  Scale  Definitions 

The  five-point  scale  below  should  be  used  to  rate  the  NCWF.  The  following  definitions  are 
provided  for  use  when  assessing  the  product.  Please  refer  to  these  definitions  when 
responding.  Also,  for  your  reference,  a  color  print  of  the  NCWF  display  has  been  provided  to 
aid  in  relating  specific  questions  to  the  components  of  the  display. 

(1)  Largely  Acceptable  -  This  response  indicates  the  product/component  being  assessed 
consistently  enhances  your  ability  to  meet  the  requirements  of  your  job;  likely  to  lead  to 
enhanced  job  performance. 

(2)  Acceptable  -  This  response  indicates  the  product/component  being  assessed  frequently 
enhances  your  ability  to  meet  the  requirements  of  your  job;  may  lead  to  enhanced  job 
performance. 

(3)  Borderline  -  This  response  indicates  that,  although  the  product/component  being  assessed 
is  adequate,  minor  improvements  would  make  it  more  helpful  in  identifying  and  forecasting 
convective  weather  and  does  not  degrade  job  performance. 

(4)  Unacceptable  -  This  response  indicates  the  product/component  being  assessed  frequently 
impedes  your  ability  to  meet  the  requirements  of  your  job:  may  lead  to  degradation  of  job 
performance. 

(5)  Largely  Unacceptable  -  This  response  indicates  the  product/component  being  assessed 
consistently  impedes  your  ability  to  meet  the  requirements  of  your  job;  likely  to  lead  to 
degradation  of  job  performance. 

NA  -  you  have  never  used  the  product/component  in  question. 


d:  ^  Other  Definitions 

You  will  be  asked  to  rate  the  NCWF  and  its  components  along  the  dimensions  of  utility, 
readability,  and  accuracy.  The  following  definitions  should  be  considered  when  answering 
questions. 

(1)  Utility  -  This  refers  to  the  usefulness  of  the  product/  component  in  meeting  job 
requirements  and  responsibilities. 

(2)  Readability  -  This  refers  to  how  readable  or  legible  the  product  or  feature  is  (for  example, 
display  clutter,  font  size,  color  coding). 

(3)  Accuracy  -  This  refers  to  the  extent  the  user  perceives  the  accuracy  of  the  identification 
and  forecasting  of  areas  of  convective  weather. 


NCWF  Products  and  Components 


Zoo  ms: 
National  zoom 


Previous  zoom 


Overlays: 

ARTCC  boundaries 
r"  Airports/ 50nm  range  rings 


Products: 

[7  Detection  field 

|7  1-hour  forecast  (blue  polygons) 

P  2-hour  forecast  (black  polygons) 


Comments  button 
coming  soon 
Product  Description  | 


Instructions:  Please  rate  the  National  Convective  Weather  Forecast  product  and  its 
components  by  circling  the  appropriate  number. 


National  Convective  Largely 

Weather  Forecast  Acceptable 

Product 

Acceptable 

Borderline 

Unacceptable 

Largely 

Unacceptable 

1 .  Initial  Detection  Fields 

a.  Utility 

1 

2 

3 

4 

5 

NA 

b.  Readability 

1 

2 

3 

4 

5 

NA 

c.  Accuracy 

1 

2 

3 

4 

5 

NA 

2.  Convective  Intensity  Color  Scale 

a.  Utility 

1 

2 

3 

4 

5 

NA 

b.  Readability 

1 

2 

3 

4 

5 

NA 

3.  Detection  Field  Colors 

a.  Utility 

1 

2 

3 

4 

5 

NA 

b.  Readability 

1 

2 

3 

4 

5 

NA 

4.  1-Hour  Forecast  Contours 

a.  Utility 

1 

2 

3 

4 

5 

NA 

b.  Readability 

1 

2 

3 

4 

5 

NA 

c.  Accuracy 

1 

2 

3 

4 

5 

NA 

5.  2-Hour  Forecast  Contours 

a.  Utility 

1 

2 

3 

4 

5 

NA 

b.  Readability 

1 

2 

3 

4 

5 

NA 

c.  Accuracy 

1 

2 

3 

4 

5 

NA 

6.  Forecast  Direction  Lines 

a.  Utility 

1 

2 

3 

4 

5 

NA 

b.  Readability 

1 

2 

3 

4 

5 

NA 

c.  Accuracy 

1 

2 

3 

4 

5 

NA 
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National  Convective 
Weather  Forecast 
Product 


Largely  Borderline  Largely 

Acceptable  Acceptable  Unacceptable  Unacceptable 


7.  50  nmi  Range  Rings 

a.  Utility  1  2 

b.  Readability  1  2 


3  4 

3  4 


5 

5 


8.  Airport  Identifiers 

a.  Utility  1  2 

b.  Readability  1  2 


3  4 

3  4 


5 

5 


9.  Storm  Height  Information 

a.  Utility  1  .  2 

b.  Readability  1  2 

c.  Accuracy  1  2 


3  .  4 

3  4 

3  4 


5 

5 

5 


10.  Echo  Speed  Values 

a.  Utility  1 

b.  Readability  1 

c.  Accuracy  1 


2  3  4 
2  3  4 
2  3  4 


5 

5 

5 


1 1 .  ARTCC  Boundary  Overlays 

a.  Utility  1 

b.  Readability  1 


2  3  4 

2  3  4 


5 

5 


12.  User  Selected  Zoom 

a.  Utility  1 

b.  Readability  1 


2 

2 


3  4 

3  4 


5 

5 


13.  Product  Update  Rate 


2  3  4 


5 


NA 

NA 


NA 

NA 


NA 

NA 

NA 


NA 

NA 

NA 


NA 

NA 


NA 

NA 

NA 
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14.  Additional  Comments.  Please  provide  any  additional  comments  you  may  have  regarding 
the  above  NCWF  products  and  components. 


15.  List  suggestions  for  improving  any  of  the  above  NCWF  products,  functions  or  displayed 
items.  Please  identify  the  item  in  the  suggestion(s). 


16.  List  other  attributes  or  components  that  you  think  should  be  added  to  the  NCWF. 


Product  Usefulness  for  Job  Tasks 


Instructions:  The  five  point  scale  below  should  be  used  to  rank  the  usefulness  of  the  NCWF  on 
each  of  the  tasks  listed.  Please  refer  to  the  following  definitions  when  responding. 

(1)  Of  Considerable  Use.  This  response  indicates  the  NCWF  had  a  significant  positive  effect 
on  decisions  and  activities  related  to  the  performance  of  this  task. 

(2)  Somewhat  Useful.  This  response  indicates  the  NCWF  had  a  positive  effect  on  decisions 
and  activities  related  to  the  performance  of  the  task. 

(3)  Borderline.  This  response  indicates  the  NCWF  had  little  to  no  effect  on  decisions  and 
activities  related  to  the  performance  of  the  task. 

(4)  Of  Little  Use.  This  response  indicates  the  NCWF  had  a  negative  effect  on  decisions  and 
activities  related  to  the  performance  of  the  task. 

(5)  Not  at  all  Useful.  This  response  indicates  the  NCWF  had  a  significant  negative  effect  on 
decisions  and  activities  related  to  the  performance  of  the  task. 


NA  -  You  have  never  used  the  NCWF  in  performing  this  task. 


Of  Considerable 
Use 

17.  Preparing  flight  plans.  1 

Somewhat 

Useful 

2 

Borderline 

3 

Of  Little 
Use 

4 

Not  at  all 
Useful 

5 

NA 

18.  Planning  fuel  quantity. 

1 

2 

3 

4 

5 

NA 

19.  Planning  flight  route. 

1 

2 

3 

4 

5 

NA 

20.  Avoidance  of  severe 

1 

2 

3 

4 

5 

NA 

convective  weather. 

21 .  Selecting  an  alternate  airport. 

1 

2 

3 

4 

5 

NA 

22.  Selecting  altitude. 

1 

2 

3 

4 

5 

NA 

23.  Monitoring  flight  conditions. 

1 

2 

3 

4 

5 

NA 

24.  Re-routing  flights. 

1 

2 

3 

4 

5 

NA 

25.  Advising  pilots  enroute  of  severe 

1 

2 

3 

4 

5 

NA 

convective  weather. 

26.  Anticipating  ground  delays. 

1 

2 

3 

4 

5 

NA 

27.  Anticipating  ATC  actions 

1 

2 

3 

4 

5 

NA 

(e.g.,  rerouting,  closing  runways, 
changing  arrival  gates) 

28.  Other 

1 

2 

3 

4 

5 

NA 

Thank  you  for  your  time  and  cooperation. 
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APPENDIX  F 


DELTA  AIRLINES  QUESTIONNAIRE  COMMENTS 


NCWF  Questionnaire  Comments 
Delta  Airlines 


Dispatcher 

Please  provide  any  additional  comments  you  may  have  regarding  the 

NCWF  and  its  components. 

1 

Biggest  negative  issue. ..product  reliability,  especially  in  high  user  demand 
times/strong  convective  activity  periods.  Also  noted  and  reported  several 
occurrences  when  forecast  model  was  not  indicating  for  VIP  3  or  better  activity. 
Reference  2  hour  forecast  contours:  never  saw  this  feature  in  action.  Greatest 
advantage  to  this  product  is  the  one-stop  feature,  which  eliminates  the  need  to 
compare  and  cross-examine  various  single  feature  products  available  from  other 
sources.  This  is  a  very  nice  product  that  just  needs  a  little  tweaking. 

2 

Perhaps  the  site  should  be  separated  into  two  parts. ..enroute  and  terminal.  For 
instance  50nm  rings  are  more  useful  enroute  rather  than  a  terminal  environment. 
More  airport  zooms  would  be  helpful  as  well.  ITWS  is  a  very  good  product  but 
not  enough  coverage.  At  times  the  max  tops  data  seems  to  be  grossly 
inaccurate.  I  find  the  forecast  contours  very  useful  particularly  when  solid  squall 
lines  contours  split  and  they  are  usually  on  target  as  far  as  forecasting  a 
developing  hole  in  the  line. 

3 

I  used  this  product  every  day  during  the  thunderstorm  season  and  was  very 
pleased  with  its  usefulness  and  performance.  There  were  just  a  couple  of 
readability  issues  with  the  graphics,  otherwise  -  great  tool. 

4 

Great  graphics  questionable  accuracy  due  to  slower  updates  at  times;  displays 
good  but  not  real  time  at  times 

Dispatcher 

List  suggestions  for  improving  any  of  the  NCWF  components,  functions  or 
displayed  items.  Please  identify  the  item  in  the  suggestions. 

1 

Color  selection  for  forecast  contour  lines:  suggest  change  the  outline  color  or 
the  background  colors.  Cyan  appears  to  work  better  on  a  dark  or  black 
background  color  in  my  experience. 

2 

Most  of  the  improvements  I  would  make  on  the  components  are  readability. 
Thankfully  it’s  been  some  time  since  I  had  to  use  your  product  that  I  can’t 
honestly  give  you  a  good  example. 

3 

We  use  24  inch  monitors  and  run  at  a  very  high  resolution.  I  would  like  to  see  a 
larger  image  option  for  users  with  very  large  screens. 

4 

More  frequent  updates  a  flight/dispatcher  in  the  area  has  no  real  use  for  a  5 
minute  or  even  2  minute  update...how  long  is  a  leg  or  turn  of  a  holding  pattern...1 
minute  I  think  and  that’s  as  long  as  we  need  to  wait  for  new  data 
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Dispatcher 

List  other  attributes  or  components  that  you  think  should  be  added  to  the 
NCWF. 

1 

Interface  with  the  ATCSCC  Severe  Weather  unit  and  display  current/projected 
reroutes.  4-frame  weather  looping? 

2 

As  previously  stated,  I  think  most  of  the  readability  issues  would  go  away  if  the 
site  contained  more  zooms. 

3 

I  would  like  to  see  actual  SWAP  routes,  which  are  in  place  by  the  ATCSCC 
graphically  depicted  on  the  map  so  that  users  could  evaluate  the  viability  of 
these  routes  collaboratively. 

4 

Fixes  around  the  airport  like  the  corner  posts  and  maybe  the  lAF  for  active 
runways 
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APPENDIX  G 


COMAIR  QUESTIONNAIRE  COMMENTS 


NCWF  Questionnaire  Comments 
Comair  Airlines 


Dispatcher 

Please  provide  any  additional  comments  you  may  have  regarding  the 

NCWF  and  its  components. 

1 

2-hour  forecasts  are  not  very  accurate. 

2 

Live  radar  loop  with  top  reports  is  just  as,  if  not  more,  useful  than  this  product. 

3 

Would  like  to  see  it  update  more  often.  The  box  for  the  zoom  is  difficult  to  see. 

4 

1  found  icing  and  storm  top  information  extremely  helpful  in  my  planning 
procedures. 

5 

Forecast  contours  are  too  often  inaccurate.  Storm  moves  east  for  10  minutes, 
then  it  moves  south  for  10  minutes  and  keeps  going  back  and  forth  between  the 
two,  instead  of  showing  the  result  of  this,  which  was  southeast. 

7 

Sometimes  shows  erroneous  information. 

8 

Product  sometimes  tracked  phantom  storms  when  no  weather  existed. 

9 

It  would  be  nice  to  have  a  setting  to  increase  updates  -  especially  on  bad 
weather  days  and  have  last  update  time  more  pronounced.  Have  had  times 
when  1  had  a  quick  look  at  the  screen  for  a  radio  call  and  began  to  give  the 
wrong  information. 

10 

Sometimes  when  there  were  many  echoes,  the  speed  and  height  indicators  were 
difficult  to  read,  as  were  the  airport  locators.  We  did  get  great  benefit  from  the 
icing  forecast. 

11 

While  flight-planning  decisions  were  not  made  based  solely  on  the  information 
provided,  it  is  definitely  a  very  useful  and  influential  tool  to  have. 

12 

When  product  was  up  it  was  fairly  useful.  Placement  of  terminals  hindered  my 
use  of  product  during  high  workloads.  Also,  lag  time  in  initial  forecast  of  new 
convective  activity  reduced  the  value  of  product  during  rapidly  changing 
conditions.  All  in  all,  a  decent  supplemental  source  of  information. 

13 

The  areas  of  forecast  icing  are  very  helpful. 

14 

NCWF  product  has,  on  occasion,  tracked  phantom  storms.  These  storms  would 
appear  on  NCWF  product  but  could  not  be  found  on  other  radar  products.  Time 
and  date  stamps  matched  on  other  products. 

15 

Product  updates  were  poor  when  the  storms  or  bad  weather  was  around.  A  lot 
of  the  time  it  was  old  data.  For  preflight  planning  this  product  could  be  very 
useful. 

16 

1 .  intensity  color  scale  -  dispatchers  needed  constant  reminders  of  color  scale 
vs.  a)  NEXRAD  and  b)  radsum  charts.  Confusion  in  interpreting  differences.  2. 
NCWF  was  available  to  the  dispatcher  75%  to  80%  of  the  day.  However,  it 
slowed  when  major  storm  systems  generated.  Information  overload? 

17 

A  couple  of  problems  1  had  was  controlling  the  enlargement  of  an  area  -  it  was 
difficult  to  control  enlargement.  1  would  have  difficulty  outlining  the  areas  1 
wanted  to  view.  Secondly,  sometimes  the  updating  would  occur  every  few 
minutes  or  more  often  and  sometimes  change  the  one  and  track  boundaries 
significantly.  1  would  then  lose  confidence  in  the  predictability.  However,  it  is  a 
great  concept.  1  really  can  use  the  forecast  to  estimate  weather  and  1  did  use  it 
to  reroute  some  aircraft.  The  ARTCC  boundary  overlays  can  be  useful. 
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Please  provide  any  additional  comments  you  may  have  regarding  the 

NCWF  and  its  components  (continued). 

18 

1  would  like  to  see  animation  on  the  detection  fields  -  possibly  the  last  2-6  hours 
of  movement. 

19 

A  couple  of  times  the  1-hour  forecast  was  changing  constantly  (every  5  minutes) 
as  a  system  approached  CVG.  On  occasion  the  system  was  down  or  not 
updating  which  caused  me  not  to  rely  on  it  as  often.  The  other  products  we  had 
to  use  were  there  then,  so  it  may  have  caused  me  to  ignore  what  this  had  to 
offer  when  it  was  working. 

20 

I  have  found  that  with  all  of  the  components  activated  the  screen  is  far  too 
cluttered  and  confusing,  the  vectors  are  useless  because  they  are  nothing  more 
than  straight  lines,  the  ARTCC  outlines  are  of  no  use  as  well. 

22 

All  in  all,  the  system  would  rate  a  success  in  my  eyes,  for  current  storm  location 
and  intensity  levels.  The  1 -hour  forecast  was  a  useful  tool  as  well,  the  majority 
of  the  time. 

24 

The  thunderstorm  prediction  algorithm  is  useful  only  to  a  point.  It  can  track 
thunderstorms,  but  only  after  45  minutes  of  level  3  data  have  been  received. 

Also,  the  storm  prediction  tracks  seem  to  follow  only  the  path  (instantaneous 
direction  and  speed)  at  the  time  of  the  prediction.  It  does  seem  to  veer  away 
due  to  other  outside  factors. 

25 

1  like  the  tops  and  speeds  given  -  very  helpful!!  1  find  the  1-  and  2-  hour  forecast 
unreliable. 

26 

1  feel  it  useful  as  a  "second  opinion".  With  an  improvement  in  its  predictions  it 
could  prove  to  be  an  extremely  useful  tool. 

28 

Zoom  is  very  difficult  to  read.  Reliability  of  system  working  would  be  needed. 

30 

Icing  is  great!  The  convective  weather  was  down  quite  often  and  didn’t  portray 
any  weather  trends  or  development.  Few  tops  were  shown,  and  movement  was 
shown  for  only  a  few  cells.  Development  and  trends  weren’t  shown  at  all. 

31 

#4/#5  -  Product  is  rated  as  such  due  to  inherent  limitations  of  trying  to  forecast 
airmass  type  thunderstorms  based  on  extrapolated  data.  1  have  personally  not 
seen  this  product  on  line  during  a  frontal-type  line,  which,  in  my  opinion,  would 
work  better. 
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List  suggestions  for  improving  any  of  the  NCWF  components,  functions,  or 
displayed  items.  Please  identify  the  item  in  the  suggestion(s). 

1 

Would  be  useful  to  show  movement  of  weather  on  this  program.  This  might  help 
us  to  better  predict  cell  or  storm  movement. 

4 

Would  be  helpful  if  2  hour  forecast  was  more  accurate. 

5 

NCWF  was  inoperable  quite  a  bit.  Needs  to  be  a  little  more  reliable. 

6 

The  zoom  function  could  be  enhanced  so  that  you  could  have  a  progressive 
zoom  in/zoom  out  function  rather  than  having  to  keep  going  back  to  the  National 
zoom  and  re-zooming  from  there. 

7 

Easier  to  zoom  in  on  a  certain  area  would  be  nice. 

9 

Bigger  print  would  be  nice  -  frontal  systems  -  great  job.  Isolated  cells  didn’t 
perform  as  well. 

10 

Would  help  if  we  could  just  select  the  airports  we  wanted  to  see.  Don’t  always 
need  them  all.  Also  would  like  to  be  able  to  select  specific  VORs  (e.g.,  FLM, 

DXV,  AIR) 

to  have  had  the  ability  to  show  the  information  in  a  loop  like  most  conventional 

radar  imagery. _ 

The  routes  on  the  map  are  useless  because  they  are  straight  line,  not  the  routes 
actually  flown.  Having  the  actual  route  and  user  definable  routes  (for  use  in 
reroutes)  would  be  helpful.  Overlays  should  be  definable  by  user.  When 
zoomed  out,  screen  clutter  was  a  problem. 


Possibly  add  lightning  strike  data.  One  and  two  hour  forecast  contours  tended  to 
vary  a  great  deal  over  several  minutes,  e.g.,  line  thunderstorms  50  mi.  west  of 
MCI  - 1  hour  contour  showed  movement  due  west.  Next  update  showed  line  of 
thunderstorms  missing  MCI  20  miles  to  the  southwest.  Storms  did  pass 

southwest  as  forecasted  but  initial  contours  showed  westerly  movement. _ 

Use  motion  in  depicting  movement.  Would  give  us  a  better  perspective  on  how 

the  storms  were  building. _ 

Point  and  click  to  expand.  _ 


It  would  be  nice  to  show  a  historical  progression  as  well  as  its  forecast 
projection.  Information  on  icing  and  turbulence  areas  as  well  as  improving  the 
heights  at  precipitation  readings  would  be  greatly  helpful.  It  would  be  great  to  be 
able  to  superimpose  the  route  NAVAIDs  on  the  map  quickly. 


Readability  of  many  items  could  be  improved  (see  items  #7  -  #10  of  ratings). 

The  display  for  routes  is  useless  since  they  are  just  straight  lines  and  not  the 

normal  routing. _ 

The  display  should  be  contained  on  one  screen  and  should  not  require  constant 

scrolling. _ 

The  2-hour  forecast  could  use  some  improvement  in  the  accuracy  field. 


If  the  1  and  2-hour  forecasts  were  more  reliable  it  would  be  a  very  nice  system. 


An  integration  with  an  ASD  would  be  extremely  useful  as  far  as  seeing  what  ATC 
may  be  doing  or  thinking. 


Loop  movement  would  be  of  great  help. _ 

A  loop  could  help  considerabi 


•  Displayed  items  -  User  selected  NAVAIDS/Airports/Points  of  reference. 

•  Function  -  For  airmass  activity,  consideration  of  other  normal  forecast  tools 
(i.e.  lifted  indexes,  presence  of  outflow  boundaries,  winds  aloft,  etc.)  would 
greatly  enhance  the  accuracy  (and  thus  usefulness)  of  this  product. 
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List  other  attributes  or  components  that  you  think  should  be  added  to  the 
NCWF. 

1 

Turbulence  and  icing  data  would  be  a  nice  option  to  access. 

2 

Radar  loop  (movement  past  hour). 

3 

An  airline’s  canned  routing,  i.e.  CVG  to  COS. 

5 

Airport  arrivals  would  be  nice  (including  along  the  chosen  arrival).  Also 
departures. 

6 

Further  development  of  the  product  to  add  forecast  of  convective  development 
would  greatly  enhance  this  product’s  usefulness. 

9 

Icing  and  turbulence  would  be  nice. 

10 

SID  and  STARS.  Jet  routes  to  the  Bahamas. 

11 

A  raw  lightning  data  loop  (strike  indicator).  A  linear  scale  for  the  base  maps  in 
nautical  miles. 

12 

Integration  with  a  regular  radar  package  would  enhance  the  product.  Also,  a 
looping  feature  would  give  some  information  of  forecast  accuracy  for  each  storm 
system  being  tracked. 

14 

Echo  speed  and  tops  could  be  larger  when  viewing  from  wider  scope  (i.e. 
national  zoom).  Good  product! 

16 

ATC  does  not  use  this  system  and  ATC  still  relies  on  delay  programs  and  ground 
stops  1  1/2  to  2  hours  prior  to  a  thunderstorm  event.  When  using  this  system  in 
planning  CDM,  ATC,  while  they  will  take  input,  tend  to  promote  knee-jerk 
reactions  to  thunderstorm  weather  and  disrupts  operations  without  cause.  What 

1  mean  by  this  is  that  ATC  will  do  whatever  they  want  and  do  what’s  in  their 
interest  as  opposed  to  what’s  right. 

18 

If  possible  it  would  be  nice  to  see  areas  of  potential  icing  and/or  turbulence 
mapped  out  on  this  map  or  an  additional  map. 

19 

A  loop  of  past  activity. 

20 

Progression  over  the  past  hour  should  be  added.  This  would  make  it  easier  to 
show  trend  of  intensities. 

22 

Something  to  indicate  movement  (immediate)  and  whether  the  storm  is 
dissipating  or  building  would  help  out  tremendously. 

28 

VOR  components  added  would  help. 

30 

Maybe  adding  winds,  directions  and  such  as  an  overlay. 
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